Predicting High-precision Depth on Low-Precision Devices Using 2D Hilbert Curves
Authors: Mykhail Uss
Ruslan Yermolenko
Oleksii Shashko
Olena Kolodiazhna
Ivan Safonov
Volodymyr Savin
Yoonjae Yeo
Seowon Ji
Jaeyun Jeong
Venue: ICML 2025 poster (2025)
Keywords: dense depth prediction
depth-from-stereo
on-device deployment
low-end devices
low-precision computations
high dynamic range
hilbert curve
quantization
Primary area: applications->computer_vision
TL;DR: High-precision depth prediction on low-end devices with low-precision calculations using depth representation as two 2D Hilbert curve components

Abstract: Dense depth prediction deep neural networks (DNN) have achieved impressive results for both monocular and binocular data but they are limited by high computational complexity, restricting their use on low-end devices. For better on-device efficiency and hardware utilization, weights and activations of the DNN should be converted to low-bit precision. However, this precision is not sufficient for representing high dynamic range depth. In this paper, we aim to overcome this limitation and restore high-precision depth from low-bit precision predictions. To achieve this, we propose to represent high dynamic range depth as two low dynamic range components of a Hilbert curve, and to train the full precision DNN to directly predict the latter. For on-device deployment, we use standard quantization methods and add a post-processing step that reconstructs depth from the Hilbert curve components predicted in low-bit precision. Extensive experiments demonstrate that our method increases bit precision of predicted depth by up to three bits with little computational overhead. We also observe a positive side effect of quantization error reduction by up to five times. Our method enables effective and accurate depth prediction with DNN weights and activations quantized to eight bit precision.

Predicting High-precision Depth on Low-Precision Devices Using 2D Hilbert Curves
Mykhailo Uss" Ruslan Yermolenko 13 Oleksii Shashko 14 Olena Kolodiazhna" Ivan Safonov
Volodymyr Savin 15 Yoonjac Yeo Scowon Ji
Jaeyun Jeong"
Abstract
Dense depth prediction deep neural networks (DNN) have achieved impressive results for both monocular and binocular data, but still they are limited by high computational complexity, re- stricting their use on low-end devices. For bet- ter on-device efficiency and hardware utilization. weights and activations of the DNN should be converted low-bit precision. However, this pre- cision sufficient represent high dynamic range depth. In this paper, aim overcome this limitation and restore high-precisio depth from low-bit precision predictions. To achieve this, we propose to represent high dynamic range depth as two low dynamic range components of Hilbert curve, and train the full- pell-preeision DNN to directly predict the latter. For on-device deploy- ment,
(a) Input (left)
(b) GT depth
(c) DispNet
(d) Our result
standard quantization methods and add a post-processing step that reconstructs depth from the Hilbert curve components predicted in low-bit precision. Extensive experiments demon- strate that our method increases the bit precision of predicted depth by up three bits with little computational overhead. We also observed pos- itive side effect of quantization error reduction by up to 4.6 times. Our method enables effective and accurate depth prediction with DNN weights and activations quantized to eight-bit precision.
Figure 1. Illustration of DispNet (Mayer al., 2016) quantization to INT8 precision (W8A8). Running inference of the quantized model on Qualcomm Hexagon DSP results in depth precision loss and quantization artifacts (c). Our method increases depth bit- width and reduces quantization error (d).
1. Introduction
Dense depth prediction has received considerable attention in recent years due to its wide-ranging applicability such areas as scene understanding (Dai et al., 2017: Yeshwanth et al., 2023), autonomous driving (Wang et al., 2019b; You et al., 2020), robotics (Wofk et al., 2019), augmented re- ality / virtual reality (Rasla & Beyeler, 2022), Internet of Things (Ignatov et al., 2022), drones (Kolbeinsson & Miko- lajezyk, 2024; Zhang et al., 2023). Usually, applications in these areas have strong limitations on hardware capabilities and energy consumption. Accurate depth estimation plays vital role in enabling robust perception and decision-making processes in these contexts. A majority of recent develop- ments in monocular (Ranftl et al., 2021; Bao et al., 2022; Oquab et al., 2024; Ke et al., 2023; Yang et al., 2024a;b; Shao et al., 2024) and binocular (Lipson al., 2021; Xianqi et al., 2024) depth estimation employ large-scale frame- works leveraging complex convolutional neural networks
'Samsung R&D Institute Ukraine, Kyiv 01032, Ukraine Department of Information-Comann Communication Technologies, Na- tional Aerospace University, Kharkiv 61070. Ukraine Faculty of Physics, Taras Shevehenko National University of Kyiv, Kyiv 01032, Ukraine 'Department of Artificial Intelligence, Kharkiv National University Radio Electronics, Kharkiv 61070, Ukraine IInstitute of Physics and Technology, NTUU "Igor Sikorsky Kyiv Polytechnic Institute". Kyiv 01032, Ukraine "Samsung Research, Seoul 06765, Republic of Korea Department Computer Science and Engincering, Konkuk University, Scoul, Republic of Korea. Correspondence to: Mykhailo Uss <m..use wmn.ussssamuung..o samsung.com.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Predicting
Depth on Low-Precision Devices Using 2D Hilbert Curves
(including Convolutional Gated Recurrent Units) (Nicolas al., 2015; Lipson al., 2021; Xianqi al., 2024), trans- former (Ranftl et al., 2021; Bao et al., 2022; Oquab et al., 2024) and diffusion (Ke et al., 2023; Yang et al., 2024a;b; Shao et al., 2024) architectures. Furthermore, this trend stems from attempts to develop foundational models to solve the task. However, the impressive progress made so far provokes a significant increase in the complexity of the proposed solutions to achieve better accuracy, which typically demands considerable computational and memory resources. Hardware limitations cause additional difficulties with on-device deployment of these complex models.
Depth maps require high bit-width for accurate repre- sentation. For example, representing depth in the range 0... 10 m with I cm accuracy requires ten bits. Depth rep- resented in eight bits suffers from the false edges on planar surfaces (Fig. lc), loss of spatial details and distortion of objects with low depth contrast. Consequently, low-end devices with low-precision computations necessarily lead to on on-device depth quality loss. To address this issue, in (Jiang et al., 2022) authors suggested keeping the final layer of the depth completion model at full moating-point precision while quantizing the weights and activations of other layers to either four-bit or eight-bit precision. This solution is not efficient because the last layer works in the highest spatial resolution and is computationally intensive. Low-end de- vices not always support mixed-preeee -ixee-precisson quantization or different bit-width for different model layers. For example, SNPE library (Qualcomm Technologies, Inc., 2025) for con- verting DNNs for inference on Qualcomm Hexagon Digital Signal Processor (DSP) supports the same bit-width for all layers (can be different for weights and for activations). The Coral Edge Tensor Processing Unit (TPU) supports only INT8 or UINT8 data. All operations with float-precision data are executed on CPU (Google, 2020). It means that the depth prediction model on devices with low-precision arithmetic will suffer from depth precision loss inddpen- dently of the quantization quality. This problem relates to the hardware limitations and cannot be solved by improving the quality of DNN quantization with PTQ/QAT.
The efficiency of DNN inference on low-end devices can be achieved by applying several strategies. One to optimize a model architecture to reduce number of parameters or latency, avoid using computationally intensive layers, use network pruning (Wofk et al., 2019; Oh et al., 2020; Liu & Zhou, 2023; Song et al., 2023). Another strategy con- sists in usage of low-precision computations (Li et al., 2021; Jacob et al., 2018). Low-bit fixed-point representation of DNN weights and activations, such as INT8, increases the number of operations run in parallel, reduces the amount of data transferred and from memory, and decreases the energy consumption of Multiply-Accumuler (MAC) op- eration (Nagel et al., 2021). When moving to lower bit- width precision, the memory amount reduces linearly, and the computational cost of matrix multiplications reduces quadratically (Nagel et al., 2021). Full-precision models are usually trained in FP32 precision and quantized to low- bit precision for on-device deployment. Quantization-aware training (QAT) emulates the quantization process during full-precision model training (Yang et al., 2019; Sakr et al., 2022; Bhalgat et al., 2020; Li et al., 2022); post-train post-training quantization (PTQ) works on trained models and seeks to convert weights into integer values with minimum quality degradation (Fang et al.. 2020; Yuan et al., 2022).
In this work, we propose to overeome this limitation by using depth representation as points on 2D Hilbert curve. This transform codes high dynamic range depth as two Hilbert curve components that can be represented in low-bit precision with minimum quality loss (Fig, 1). Our method adds an on-device post-processing step that converts low-bit precision Hilbert components to higher precision depth. The post-processine does not involve addition or multiplication operations and can be implemented ble (LUT). The main difference of our method is that for improving quality of DNN inference on device we consider not only DNN architecture modification, training process (like QAT), efficient weights and activations quantization (like PTQ) but also structure of the signal predicted by the DNN. Our method does not modify quantization process by itself and is built on top of existing quantization methods (either PTQ or QAT); progress of these methods in specific domains or for specific architectures will bring improvement in the performance of our method as well.
simple lookup ta-
Running inference of quantized models comes at the cost of additional low-precision computation errors and data pre- cision loss (Nagel et al., 2021). Considerable research has been devoted towards reducing quantization error by DNN's architecture modification (Sheng et al., 2018; Wang et al., 2020), quantization techniques improvement (Sakr et al., 2022; Bhalgat et al., 2020; Li et al., 2022: Nagel et al., 2021: Fang et al., 2020; Yuan al., 2022), adaptation to specific hardware (Shlezinger et al., 2019; Kiyama et al., 2019; Wang et 2019a). The problem of data precision loss is largely unexplored, presumably because it does not pose a problem for models predicting RGB images, which are naturally presented as three eight- bit channels. Nevertheless, precision loss becomes an issue for High Dynamic Range images. This prevents the use of efficient homogeneous bith quantization and forces resort to less efficient mixed quantization schemes (Borrego-Carazo et al., 2021).
Our main contributions can be summarized as follows:
(a) We propose a novel method for high-quality depth predic- tion on devices with low-precision arithmetie. The method consists of representing depth as 2D Hilbert curve compo- nents, training a full- precision model to predict 2D Hilbert
2

---

Predicting High-preeiiioo Depth Low-Precision Devices Using 2D Hilbert Curves
curve components, and reconstructing sich-precision depth from low-precision Hilbert curve components.
Mos 2
6+c
(b) We evaluate the proposed method and demonstrate that the modified model quantized to W8A8 format (8-bit weights and activations) can predict depth with similar or better quality than the original model in W8A16 16 format (8-bit weights and 16-bit activations). At the same time, the inference time and power consumption of the modified W8A8 model with pest-processing are 1.5 times less than that of the original W8A16 model.
ID
2D
20 remapped back to ID
(c) We observe that the application of the proposed method has additional positive side effect on the quantization quality, specifically quantization error reduction by up to 4.6 times.
Figure 2. Idea illustration. ID range is quantized to N = 8 values 9o 0... 11 marked by white circles. The ID range mapped to 2D curve shown in red color. Both x and axes are also quantized into N 8 values yielding 64 2D points. Among them. 36 points lie on the curve (shown in blue color). Mapping the 2D curve back the range results in different quantization values. Quantization error has effectively been reduced by the factor equal the curve length 35/7
2. Method
In this section, we introduce our method that allows high- precision depth prediction on devices with now-precision computations. The methods rests on depth representation as two components of a 2D Hilbert curve. On-device, these components are predicted in low-bit precision on TPU or NPU and then converted high-bit precision depth us- ing a simple post-processing algorithm
the complexity of post- proseprooessiig run on CPU. In the solu- tion proposed in (Jiang et al., 2022), the last layer of the depth completion model in float precision. This version is ineffective because it requires passing to the CPU large amount of data from the penultimate layer and processing the high-resolution data associated with the last layer.
CPU. In the following subsections, we describe the method in general (Sec. 2.1), discuss properties of the Hilbert curve and its utility for depth coding (Sec. 2.2), explain implementation details (Sec. 2.3), discuss expected change of quantization error (Sec. 2.4). Lastly, we present the loss function for training the modified model (Sec. 2.5).
Our idea is to represent g as point on 2D parametric curve (x(g), v()) with length L >>> 1, where both xx)) and v(z) are bounded in the [0, 1] range. The full-precision DNN is trained to directly predict x(4) and y(q). When running inference on DSP, the value of q calculated from x and y values predicted in b-bit precision. The parametric curve of length L. will pass through approximately T. 2 discreet points (x(g), v(g)) effectively increasing precision of recon- structed value of 9 by log, L bits from ò to b+ L (Fig. 2). In this implementation, the amount of data to transfer from DSP to CPU increases only twofold, post-processing on CPU is simple and implemented using LUT. We will later refer to unmodified DNN as original model and to model predicting Hilbert components (and post-processing step when it is clear from the context) modified model.
2.1. precision depth on a low-precision device
Let us consider a DNN that predicts some quantity q bounded in the range J0, 1]. quantized model runs on low-end device that has an efficient DNN inferencing module (in our case, it is DSP) with b-bit output data rep- resentation and a general-purpose purpose CPU with full- precision arithmetic. The DSP outputs value Quamb in bit precision (e.g. INT8). The error between and Bouess is called the quantization error. We denote the standard deviation (SD) of this error as Guuam.
We seek to increase the bit-width of the predicted value of q with minimum computational overhead. For the depth prediction case, corresponds to normalized depth or nor- malized disparity, depending on the DNN design.
On- devicv deployment of a depth prediction DNN consists of the following steps (Fig. 3): (a) training the full-precision model to directly predict the Hilbert curve components of depth, (b) applying standard quantization methods (either QAT during training or PTQ trained model), (c) running inference of the modified quantized model on-device and obtaining Hilbert curve components in low-bit precision, (d) applying post-processing to Hilbert curve components and reconstruct depth in higher-bit precision.
Due to DSP limitations, the bit-width of the model output cannot be increased, so we can operate only with the number and structure of the output channels. These channels should be transferred from DSP CPU and used to reconstruct q in higher bit precision. To make this scheme effective, the following key problems need to be solved: (1) minimize the amount of data transferred from DSP to CPU, (2) minimize
Selection of function (z(g), v(4) is crucial for successful implementation of the proposed idea.
3

---

Predicting High-preeiiion Depth Low-Precision Devices Using 2D Hilbert Curves
points of the curve of the previous order. We will later re- fer to the curve order as p. The approximating polygons for curves with orders - 4 are shown in Fig. 5, In or- der to avoid boundary effects, we scale each curve so that it fits into the square {(x, y) k S x; I 1 - k), where k = 0.1. In this case, the length of the p-th order curve is Lp - (2 + 1) 2k:). The length of an edge of p-th order approximation polygon equals hp = (1 22p 1). This value also defines the minimum distance between points of different parallel edges of the Hilbert curve ap- proximation polygon (Fig. 5).
WEAA
Device DSP
pair
PTO/CAT quantizd stereo matching model
Device CPU
Hilbert curne decoding
High- depth
Low-bit components depth
Figure 3. Scheme of the proposed method's inference pipeline on device.
2.3. Forward and backward transformations
For full-precision and quantized models, predicted values (x, y) will not have an exact match with any (e(2),(())
2.2. Selecting a suitable parametric curve
To convert arbitrary point (æ, y) back to ID value, we link
Let us consider the simplest function: zz()) = L
(r, y) to the closest point on the Hilbert curve:
255 2 v(4) = - xg)), where [.] denotes the floor function. In this case, z(q) coarse representation of q and y(g) adds fine details (see example in Fig. 4e and 4f). The length of this curve is 256, and it allows recon- structing g in INT16 precision from 20(4) and y(q) in INT8 precision. However, structure of y(q) is unfriendly for the full-precision model training: it has multiple sharp transi- tions from zero to one and from one to zero. During our experiments we found that this curve degrades full- full-preeisio precision model training and is very susceptible to quantization errors.
azy
arrginnnleee x(4), y v(1)l-- qe(0.1]
(I)
Building forward (ID 2D) and backward (2D - ID) mappings for Hilbert curves of arbitrary order is based on iterative algorithms (Bader, 2012). Because work with one specific curve order, faster transformation can be implemented with LUTss We build two LUTs for corre- sponding mappings. The first LUT built using bilinear interpolation of the nodes of the low-order Hilbert curve. This LUT allows us to get (I(ạ), y(4)) for the given 9. An example of this transformation applied the ground truth (GT) disparity map is shown in Fig. 4a-4b. To map 2D values (z, y) (Fig. 4c-4d) back to the ID representation, we use a second LUT that is built based on Eq. (1). In our experiments, this LUT has a size of 256 by 256 elements.
Let us discuss the desired properties of the curve (x1(),v(a)
1. Continuity: small changes in should result in small changes in both z(g) and y(4). For depth coding, this property ensures that x(4) and y(q) preserve spatial smoothness and do not introduce new depth disconti- muities.
2.4. Quantization error transformation
2. Non-seff-inteessecton: the curve should preserve one- to-one correspondence between q and (I(4), v(q))-
For the original model, the quantization error is measured at the model output level. For our method, this error is measured after post-procsssing of Hilbert components and accounts for Hilbert components quantization error and its transformation during post-processing. We denote SD of this error as quant and SD of Hilbert components quantiza- tion error as o Ssyquant-
3. Boundedness and self-avoidance: the curve should cover the unit square uniformly and avoid close points (x(g1), (q))) and (x(g2), y(92)) for distant 9ı and q2-
Curves with the desired properties are known space- fille--illing curves or Peano curves (Sagan, 2012). For the goals of this work, we found one particular curve, namely the Hilbert curve (Sagan, 2012; Bader, 2012), to be the simplest and the most flexible option. We describe this choice in more detail in Appendix A.
Locally, Eq. (1) reduces to the projection of (x, v) to the closest Hilbert curve edge, discarding x or y component (depending on the edge orientation), compressing the re- maining component L times. We discussed above that this transform adds log, L bit to q precision. In addition, Hilbert components quantization error is compressed L times yield- ing ā quant Oxy T..
The Hilbert curve is continuous fractal space-filling curve that is constructed as a limit of piece-wise linear curves (Bader, 2012). The Hilbert curve starts from a sin- gle point in the middle of the unit square. Each subse- quent curve order is produced by replicating and linking
Quantization error compression takes place if (a) Hilbert components quantization error is independent between chan- nels and identically distributed, and (b) if Txygamm is small
4

---

Predicting Hlihhprrecision Depth Low-Precision Devices Using 2D Hilbert Curves
(d)
nn)
Figure 4. Illustration of disparity transforms: (a) disparity map: (b) mapping to 2D with second order Hilbert curve; (c. d) and y components of the Hilbert curve; (e, coarse and fine details disparity map. Fine details (f) are the least significant byte of disparity (a) represented in 16-bit format. High-froquene oscillations make it appear different from the original disparity and difficult to predict by a DNN model.
Proposed DNN modification
Ingeut
240:00
Hillet Curve
Outpot
Gasadion (ppiea)
Disparky map
Storeo pair
20
Hilber Guevel
Losa function
Figure 5. Hilbert curves for orders 1,2,3,4 4 (from left to right). Every order is formed by the replacement of every node by an elementary 3-segment sequence.
Figure 6. DispNet (Mayer et al., 2016) model modification re- quired by the proposed approach. The input RGB stereo pair is processed by the encoder-decoder network from original model. Features from the encoder-decoder are fed to the optional Gaussian noise layer, followed by two identical heads for Hilbert curve components. At the post-processing stage, Hilbert compo- nents are converted to the final disparity map.
enough for Hilbert components quantization error be smaller than hp. Both original and modified models have similar architecture, solve the same task, are trained the same data, quantized by the same procedure. We can additionally assume (c) that quant Oxquuat- If this addi- tional condition is valid then our method will reduce quan- tization error by L times compared to the original model: 0 qum
where Zor and Beer are GT values for Hilbert curve compo- nents calculated from GT value far. and a is hyperparameter. The additional component A is calculated as follows:
quait
We satisfy condition (b) by proper selection of the Hilbert curve order. For the stereo matching, we experimentally found that = 2, are suitable choices, providing curve length 4-7.2 and bit-width increase by 2-2.85. Assumptions (a) and (c) can only be verified experimentally for a specific DNN architecture. We discuss their validity in detail in Sec- tion 3.5. Notice, that bit-width increase will take place even if conditions (a) and (c) not fulfilled. These additional conditions are only needed for potential quantization error reduction.
Ah (Fars Bers v) (Fer - xx) (Var y)² + B-r 2g (3)
where Tay - |1 x((o))())) ) y y(( and ,3 is additional hyperparameter. The Hilbert curve loss component serves two goals: it penalizes the distance between GT and pre- dicted points in the 2D representation; it penalizes deviation across the Hilbert curve and forces the model to predict only points that belong to the curve. We set a = and 3 = 25 in our experiments.
3. Experiments
2.5. Model and Loss Function Modification
Two models are selected for the stereo matching experiment: DispNet with the original architecture proposed in (Mayer et al., 2016) and Dense Prediction Transformer (DPT) (Ran- ftll et al., 2021) with Mobile ViTv3-S (Wadekar Chaurasia, 2022) an encoder. In all experiments, the models' input size is 384 x 512 pixels and the output size is 192 256 pixels. All modifications of the DPT model are described in detail in Appendix B. The head predicting Hilbert curve components for the DPT model is the same as for the Disp- Net but includes an additional convolutional and up-sample layer in each branch to adapt to feature shapes of the DPT decoder. We found that injecting small amount of Gaus-
To implement the proposed approach, DNN with one head predicting quantity should be modified to have two heads predicting Hilbert curve components x and y. This modification for the DispNet model (Mayer et al., 2016) is illustrated Fig.
The loss function for the proposed method is composed of two terms: depth loss A(ger: Qzy) (Hu et al., 2019) and additional term ( Mor, y) that assures model con- vergence to the Hilbert curve-based representation:
fall A(ger; (ry) + a AH (Fer, Mer, R. y);
5

---

Predicting High-precision Depth on Low-Precision Devices Using 2D Hilbert Curves
sian noise at the beginning of the Hilbert components head improves quantization of modified models with the SNPE library and has no effect on unmodified models. Experi- mental results for modified models are presented with a Gaussian noise layer with SD equals 0.02.
ning windows, Cyj and Eÿ are vectors of DCT coefficients of GT and predicted depth maps for frame i and window j. Experiments show that So calculated in 4 4 window is sen- sitive to depth blurring and INT8 quantization artifacts. The value of Se close to unity (the maximum possible value) indicates high-quality depth maps with sharp edges and absence of artifacts in homogeneous areas.
3.1. Implementation Details
For stereo matching models training we adapted Scan- Net v2 (Dai et al., 2017) dataset in the following way. Train- ing, validation, and test data are rendered from meshes pro- vided for each ScanNet v2 scene using PyRender 1.0.1.45 library. The camera poses for the left camera are fixed to the same values specified in ScanNet v2. The right camera is shifted by 60 mm along the axis x to form the horizontal baseline. Intrinsic parameters for the left and right cameras correspond to ScanNet data: pinhole camera with f= = fg = 577.87, Cr = 320, Cy = 240. The split of the dataset into training and test parts corresponds to the official ScanNet v2 split.
To characterize quantization error, we use SD ô between raw outputs of full full-precision and quantized models mea- sured in robust way as Scaled Median Absolute Devia- tion (Rousseeuw & Croux, 1993).
3.3. Analysis of W8A8 Models
We trained DispNet and DPT models using p 1,2, 4. These models will be later referred to as hpDispNet and hpDPT, respectively. We observed that float-precision mod- els hpDispNet and hpDPT learn to predict points close to the Hilbert curve. Depth prediction accuracy of modified models is similar to the original models w.r.t. all metries.
All models were quantized using SNPE SDK v.2.24 and tested on Samsung S24+ device with Qualcomm Snap- dragon 8 Gen 3 processor and Hexagon DSP. We com- pare models in FP16, W8A16 and W8A8 formats running on Hexagon DSP. Power consumption is measured with Monsoon Solutions FTA22D Power Monitor in power save mode.
Next, compare the original and modified models in W8A8 format running on CPU and DSP. The difference is that W8A8 model runs on CPU in dequantization mode using full precision arithmetic, while on DSP inference is done in lower-precision arithmetic. Quantitative results are presented in Table 1. For the original DispNet model, quan- tization leads noticeable quality degradation on both CPU and DSP. On CPU degradation is seen for S. metric that drops from 0.86 to 0.68 reflecting loss of spatial details.
3.2. Evaluation Metrics
We characterize the quality of predicted depth using stan- dard metrics (Eigen et al., 2014; Geiger et al., 2012): mean absolute relative error (Abs Rel), root mean square error (RMSE), end- pointtont-eroo (EPE), and DI.
Modified models for all p perform better than the original ones; the best result achieved for p = 3. As shown in Fig. 7, the quantized h3DispNet model retains the ability to predict points on the Hilbert curve for both CPU and DSP inference. The h3DispNet model on CPU shows almost the same quality as the FP32 model and outperforms the original DispNet w.r.t. all metrics. On DSP, the quality drop of the original DispNet is more significant: Abs Rel increases from 1.01 to 2.03, and S. decreases from 0.86 to 0.58. The h3DispNet model compensates this drop almost completely reaching Abs Rel 0.93 and S. 0.81.
Pixel-level errors themselves are not sufficient to character- ize quantization artifacts in the predicted depth maps. For example, INT8 depth representation has minimal impact on the Abs Rel metric. To address this issue, we experi- mented with the SSIM metric (Wang et al., 2005; Rouse & found it barely affected by
Hemami, 2008). However, quantization artifacts. Therefore, we propose using cosine similarity (Lahitani et al., 2016) between discrete cosine transform (DCT) (Ahmed et al., 1974) coefficients of GT and predicted depth maps. For this, n X n DCT is applied in a scanning window manner to both GT and predicted depth maps. The DCT coefficients are flattened to vector representations and zero coefficients are discarded. Cosine similarity between flattened vectors is calculated at each scanning window position and then averaged:
For the DPT model, the situation is similar, but the quality drop for the quantized model is more significant on both CPU and DSP. On CPU, the h3DPT model performs better than the original FP32 DPT model. On DSP the best result shows h2DPT model with Abs Rel improving from 4. 18% to 1. 12% and S. increasing from 0.52 to 0.62 as compared to the original model. The curve of the third order provides the best compromise between Abs Rel and S, improvement for both DispNet and DPT models. This conclusion holds when RMSE, EPE or DI are taken into account.
N
M
Cij
Cij
Sc =
(4)
cij
|alll
Qualitative results for the h3DPT model are illustrated in Fig. 8. Reduction of quantization error between the original
where Ni is the number of frames, M is the number of scan-
6

---

Predicting sigh--peecision Depth Low-Precision Devices Using 2D Hilbert Curves
Abs Rel,
RMSE, Px.
EPE, px
DI, 4
Model
FP32
FP32
W8A8
FP32
W8A8
FP32
W8A8
FP32
W848
CPU
DSP
CPU
CPU
DSP
CPU
CPU
DSP
CPU
CPU
DSP
CPU
CPU
DSP
CPU
DispNet
1.01
2,03
1.15
1.12
2.24
1.10
0.86
0.58
0.68
0.29
0.69
0.31
1.81
5.35
1,73
DiDispNet
1.06
1.50
1.12
0.97
1.09
0.97
0.86
0.67
0.79
0.27
0.35
0.29
1.27
2.25
1.27
N2DispNet
0.85
0.98
0.88
0.90
0.94
0.91
0.87
0.75
0.83
0.22
0.25
0.23
1.01
1.26
1.02
beDispNet
0.88
0.93
0.87
1.00
1.03
1.00
0.87
0.81
0.86
0.24
0.24
0.24
1.25
1.26
1.25
NeDispNet
1.02
1.02
1,24
1.39
hIDPT
0.70
1.48
0.78
0.88
1.41
0.89
0.88
0.54
0.88
0.20
0.41
0.22
1.01
2.77
1.03
12DPT
0.71
1.12
0.72
0.91
1.02
0.91
0.88
0.62
0.88
0.20
0.20
1.07
1.27
1.08
13DPT
0.55
1.35
0.63
0.80
1.33
0.80
0.90
0.90
0.15
0.32
0.17
0.78
1.28
0.78
h4DPT
0.74
1.27
0.76
0.94
1.38
0.94
0.87
0.73
0.86
0.21
0.32
0.21
1.14
1.62
1.14
Table Metrics of DispNet, hpDispNet, DPT and hpDPT models. All metrics are presented for FP32 model (CPU inference), W8A8 model running DSP and W8A8 model running CPU (dequantization mode). The best results DSP are in bold font. Results for the original DispNet and DPT models are marked in grey color.
Y
(a) GT depth
(b) DPT depth
(c) h3DPT depth
0.2
0.0
0.5
(a) W8A8 model on CPU delegate
(d) Image
GT DPT]
(0) [GT h3DPT]
S22
Figure 8. Quantization errors of DPT and h3DPT, W8A8, DSP.
(b) W8A8 model DSP delegate
(a) DispNet
(b) h2DispNet
Figure 7. 2D histogram of h3DispNet W8A8 model output for CPU and DSP delegates.
Figure 9. Effect increased bit-width on the quality depth prediction for homogeneous areas DSP.
model (Fig. 8h) and the modified model (Fig. 8c) is very be
significant
maps in Fig. 8e and Fig. 8f. We also observe from Fig. 8c and Fig, 8b that h3DPT demonstrates better spatial details than the original model. This effect caused by increase of effective number of bits for depth map coding by log, L bits (approximately by 2.85 bits h3DPT resulting in INTIO-INTI] precision). Indirectly, this effect is characterized by increase S, value with p (see Table 1). Visually, bit- bit-precision increase is well seen for homogeneous planar areas as illustrated in Fig. 9.
study more detail in Section 3.5. Effect of the improved quality of depth prediction on scene mesh fusion discussed in Appendix D and additional examples of predicted depth maps given in Appendix E. Additional experiment KITTI 2012 dataset is described in Appendix C.
3.4. Comparison of FP16, W8A16 and W8A8 Models
Compared to the original model, the overhead of our method consists of three parts: (1) increased model complexity, (2) additional data transfer from DSP to CPU, (3) post- processing of Hilbert components. We found that this over- head (= 14%) is significantly smaller than the performance gain from using W8A8 models instead of W8A16 16 FP16
As one can see from these results, the proposed method not only solves the main problem of bit-width increase but also significantly reduces the level of quantization error of depth prediction. The latter positive side effect and we will

---

Predicting High- Iligh-precision Depth Low-Precision Devices Using 2D Hilbert Curves
Precision
Abs Rel,
EPB, px
DI,
Sc
T.
s/inf.
trecust Ac-os the curve
1030
Nes
250
1203
FF32
1.01
0.29
1.81
0.858
1933
FP16
1.50
0.37
1.80
0.855
19.54
19.52
W8A16
1.78
0.63
5,22
0.798
18.7
12.3
023
130
0.585
900
Ours, W8A8
0.93
0.24
1.26
0.807
12.0
8.7
DPT
50
FP32
0.889
TPI6
1.14
0.27
0.97
0.884
$4.1
110.5
-5902
0.02
0.53
0.825
dimarsaoriinn
Erar: dimerstortas
WSA8
4.16
1.03
5.76
0.520
26.7
28.3
Ours, W8A8
1.35
0.32
1.28
0.697
30.4
29.7
(a) Distributions of errors
(b) Disparity errors distribu- tion for h3DispNet and Disp- Net models.
across and along the Hilbert curve.
Table On-device performance of the original and modified mod- els including runtime (7) and power consumption (P). Measure- ments for our method include data transfer from DSP CPU and Hilbert components post-processing on CPU. The overhead of our method is in runtime and power consumption increase between modified and original W8A8 models (compare lines marked by orange and green colors). Models in FP32 format are run on CPU: models in FP16, W8A16, W8A8 formats are run on DSP.
Figure 10. Quantization errors distributions for the h3DispNet model. Disparities d are calculated from Hilbert components and and normalized to [0, range.
by the curve length L. Quantization error in Hilbert curve- based output representation is significantly smaller than expected if the original model errors are scaled L times, As result, in disparity space, the quantization error of the modified model is significantly smaller than that of the original one (see Fig. 10b): ā 5.65-10 and 17.66- 10- respectively. Thus, we obtained quantization error reduction by #3.1 times on CPU, On DSP this gain is #=4.6 times. Similar effect is observed for the DPT.
models. Detailed profiling results are presented in Table 2 for models with the curve order 3.
The modified DispNet model in W8A8 format shows better quality than the original model in W8A16 16 format while simultancously reducing energy consumption by 35% and latency by 30%. Compared to the original model in W8A16 format, the modified DPT model in W8A8 format shows significantly better Abs Rel metric and only slightly worse S.. It also lowers energy demands by 34% and processing time by 54%.
In order to bring further understanding, we applied Hilbert curve-based output representation to the Human Pose Es- timation (HPE) task (see Appendix F). For the ResNet- RS (Bello et al., 2021) encoder and keypoints representation as heatmaps, the effect is similar to stereo matching but with a stronger along-the-cune error reduction on DSP is 2.69 times for p 3 curve. For direct keypoints regression along-the- thee-the- correlation is so strong that quantization error reduction is not observed. For both stereo matching and HPE, we found that the magnitude of the across-the-curge quantization error is similar to the quantization error of the original model.
correlation. Quantization
3.5. Analysis of Quantization Errors Reduction
For the detailed analysis of quantization error reduction, we selected the h3DispNet model. To exclude precision loss in- fluence, we perform analysis for W8A8 model run on CPU. The difference between the original and modified model lies in the number of channels utilized: the relationship between the quantization errors among two Hilbert compo- nents requires further examination. Our initial analysis in Section 2.4 was made under the assumption of independent quantization errors for the Hilbert components. However, assessment using real data demonstrated that this hypothesis does not hold universally, as correlation has been discov- ered along the Hilbert curve. Quantitatively, the distribution of quantization errors along-the-curve wider than that across-the-curve (Fig. 10a).
4. Discussion and Limitations
The proposed method is designed with the main goal of increasing the bit-precision of the predicted depth on de- vices with low-bit precision output. Apart from this, we observed a positive side effect consisting of a significant reduction of the quantization error. The effect reveals itself in different degrees for the depth prediction and human pose estimation with heatmaps; is not observed for the human pose estimation with direct keypoints regression.
Across-the-curve errors are mainly nullified in the post- processing, along-the-curve errors are compressed L times and define the level of errors of the target quantity, that is disparity. To compare errors in the same scale, we added in Fig. 10a disparity error of the original model multiplied
Our experiments show that the quantization error of Hilbert components can be correlated and have a large SD along the Hilbert curve. This effect manifests itself strongly for the human pose estimation with direct regression and only
8

---

Predicting High-precision Depth Low-Precision Devices Using 2D Hilbert Curves
slightly for the stereo matching models. Such correla- tion indicates that instead of learning independent paths for predicting Hilbert curve components, the modified model learns an internal representation of the target quantity (for example, disparity, depth or keypoints) and converts it into Hilbert components at the final model layers. In this case, the modified model predicts the target quantity with the same quantization error as the original model, this error un- dergoes forward transform Hilbert representation (inside the model) and backward transform to the target quantity (at the post-procces proseprrcessing step). As result, the quantization error level remains the same, and the effect of our method application is only in bit-width precision increase. This is the case for the HPE with direct regression.
5. Conclusions
In this paper, we proposed a novel method for high dy- namic range depth prediction on devices with low-precision arithmetic that exploits depth representation as points on 2D Hilbert curve. This representation essentially codes the high dynamic range depth as two low dynamic range Hilbert curve components. The depth prediction model is trained to directly predict two Hilbert curve components that are calculated on-device in low-bit precision and used to reconstruct depth in high-bit precision. Apart from increas- ing bit-precision, our method reduces quantization error by factor of up to 4.6. Experiments demonstrate that for the stereo matching task our method reconstructs depth in INTIO-INTI 1 bits for a model quantized in W8&8 format and with quality similar to or even better than the original model quantized in W8A16 format. In this manner, depth can be predicted on-device 1.4-2 times faster and with 65% of power consumption without sacrificing its quality. The proposed approach is beneficial for on-device application of different dense depth prediction methods including monoc- ular and stereo depth prediction, completion, depth quality enhancement, and depth inpaint- ing. Future efforts need to be made to understand and fully explore the effect quantization error reduction.
If both x and y are predicted by independent paths inside the model, the quantization errors of the Hilbert compo- nents are not correlated and have similar level as for the original model. This error is compressed L. times at the Hilbert components post-processing stage. As result, our method allows both reducing quantization error and increas- ing bit-width precision as has been manifested by our stereo matching experiment. If the source of correlation in the along-the-curve direction can be found and eliminated, the proposed method could be used to reduce quantization error in cases when bit-precision is not an issue.
View--teew-eo depth
While our method improves the quality of predicted depth, it requires retraining of the full-precision model. In addi- tion, our method requires that errors of the quantized depth prediction model are bounded, since large values of the quantization error violate one-to-one correspondence be- tween the depth values and the points the Hilbert curve. The next step involves joint usage of QAT and our method and moving to higher dimensions where a multidimensional Hilbert curve of the same length fills the space less densely.
Impact Statement
This paper proposed an innovative approach for increasing bit- bit- precision of quantized depth prediction models on de- vices with low-precision arithmetic. This paper contributes to the advancement of quantization techniques for dense prediction tasks for devices with limited resources. While there are possible social implications that arise from our work, none of them are particularly relevant in this context.
We present results for stereo-matching models that predict disparity in one step. The recent models showing the best quality for stereo matching (Xu et al., 2023; Xianqi et al., 2024) and monocular depth prediction (Shao et al., 2023) incorporate iterative disparity refinement using GRU or sim- ilar recurrent units. In these architectures, the predicted dis- parity is refined over multiple iterations by adding a small correction signal to the initially predicted disparity map. In- tegration of the proposed idea of disparity representation as two Hilbert curve components into iterative models differs significantly from one-stage models like DispNet and DPT. While we experimented with modification of the output disparity representation, modification of iterative models requires integration of Hilbert components inside the model at multiple places. We leave this interesting problem for future work.
References
Ahmed, N., Natarajan, T., and Rao, K. R. Discrete cosine transform. IEEE Trans. Computers, 23(1):90-95 1974. doi: 1..1109/-C..974.22339
Bader, M. Spuce-filling curves: an introduction with ap- plications in scientific computing. volume 9. Springer doi: 10.1007/
Science & Business Media, 2012.
978-3-642-31046-1.
Bao, H., Dong, L., and Wei, F. BEiT: BERT pre-training of image transformers. In Int. Conf. Learn. Represent., 2022. doi: 00448500/rxxx.2206.0222
Bello, I. Fedus, W., Du, X., Cubuk, E. D., Srinivas, A., Lin, T.-Y., Shlens, J., and Zoph, B. Revisiting ResNets: Improved training and scaling strategies. In Int. Conf. on Neural Information Processing Systems, pp. 22614
9

---

Predicting High-precision Depth Low-Precision Devices Using 2D Hilbert Curves
2262, Red Hook, NY, USA, 2021. Curran Associates Inc. ISBN 9781713845393. doi: 10.5555//550266.354111
X., Jiang, J., Shi, X., Xu, D., Li, Y., Wang, X., Lei, L., Zhang, Z.,, Wang, Y., Huang, Z., Luo, G., Yu, G., Fu, B., Li, J., Wang, Y., Huang, Z. Cao, Z., Conde, M. V., Sapozhnikov, D., Lee, B. H., Park, D., Hong, S., Lee, J., Lee, S., and Chun, S. Y. Efficient single-image depth estimation on mobile devices, mobile ai & aim 2022 chal- lenge: Report. Eur. Conf. Comput. Vis., pp. 71-91, 2022. doi: (0000/97--30031-25550
Bhalgat, Y. Lee, J., Nagel. M.. Blankevoort, T., and Kwak, N. LSQ+: Improving low-bit quantization through learn- able offsets and better initialization. In JEEE Conf. Com- put. Vis. Pattern Recog., pp. 2978-2985, 2020. doi: (0.100//CVRRW55488220000000
Borrego-Carazo, J., Ozay, M., Laboyrie, F., and Wishey, P. A mixed quantization network for computationally efficient mobile inverse tone mapping. The British Machine Vision Conf., 2021. doi: 10.48550/mXXiv2220..5555 005544
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. Quan- tization and training of neural networks for efficient nnleger-arilmmetic-only inference. In IEEE Conf. Com- pul. Vis. Pattern Recog. pp. 2704-2713, 2018. doi: 10.1 10.1109/CVPR.22188.000
Cabon, Y., Murray, N., and Humenberger, M. Virtual KITTI 2. arXiv, 2020. doi: 10.48550/axXiv.200.1107
Curless, B. and Levoy, M. A volumetric method for build- ing complex models from range images. In Proc. of An- mual Conf. on Computer Graphics and Interactive Tech- niques, SIGGRAPH '96, pp. 303 - 312, New York, NY, USA, 1996. Association for Computing Machinery. ISBN 0897917464. doi: 1e..1144/77770222777
Jiang, X., Cambareri, V., Agresti, G., Ugwu, C. I., Simon- etto, A., Cardinaux, F., and Zanuttigh, P. A low memory footprint quantized neural network for depth completion of very sparse time-of-flight depth maps. In IEEE Conf. Comput. Vis. Pattern Recog. Worksh. pp. 2687-2696, June 2022. doi: 10.1100/CVPRW55347222200000
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., and Nießner, M. ScanNet: richly-annotated 3D recon- structions of indoor scenes. In JEEE Conf. Comput. Vis. Pattern Recog,, 2017. doi: 10. N0.1109/CVPR.2017.206
Ke, B., Obukhov, A., Huang, S., Metzger, N., Daudt, R. C.,, and Schindler, K. Repurposing cerfuusion-basedd image generators for monocular depth estimation. IEEE Conf. Comput. Vis. Pattern Recog., pp. 9492-9502, 2023. doi: 10. N0.1109CVPRS2233.2222.000
Eigen, D., Puhrsch, C., and Fergus, R. Depth map prediction from a single image using multi-scale deep network. Adv. Neural Inform. Process. Syst., pp. 2366-2374, 2014. doi: 10.5555/2969033.299999
Kiyama, M., Nakahara, Y., Amagasaki, M., and lida, M. A quantized neural network library for proper implementa- tion of hardware emulation. In Int. Symp. on Computing and Networking Workshops (CANDARW), pp. 136-140, 2019. doi: 10. 101199AADDAAR..209000....
Fang, J., Shafiee, A., Abdel-Aziz, H., Thorsley, D., Geor- giadis, G., and Hassoun, J. H. Post-training piecewise linear quantization for deep neural networks. In Eur. Conf. Comput. Vis., pp. 69-86. Springer, 2020. doi: 10.1007/978-3-030-555588
Kolbeinsson, B. and Mikolajezyk. K. DDOS: The drone depth and obstacle segmentation dataset. In IEEE Conf. Comput. Vis. Pattern Recog. Worksh. pp. 7328-7337, 2024. doi: 10. 22110//CVRWW333222222444
Geiger, A., Lenz, P., and Urtasun, R. Are we ready for autonomous driving? The KITTI vision benchmark suite. In IEEE Conf. Comput. Vis. Pattern Recog., pp. 3354- 3361, 2012. doi: 10.1109/CVPR.2222..24400
Lahitani, A. R., Permanasari, A. E.,, and Setiawan, N. A. Cosine similarity to determine similarity measure: study case in online essay assessment. In Inf. Conf. on Cyber and IT Service Management, pp. 1-6, 2016. doi: 10.1109/ CITSM.2016.757757.
Google. Coral Edge TPU documentation, 2020. URL / modelssinntro/Aoodel-- heg.asa-ettttoooolleeeeeeeeeeeee
Hu, J., Ozay, M., Zhang, Y., and Okatani, T. Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries. In IEEE Winter Conf. on Applications of Comput. Vis. (WACV), pp. 1043- 1051, 2019. doi: 10.1 10110//A/22001.0000
Li, G., Xue, J., Liu, L.. Wang, X., Ma, X., Dong, X., Li, J., and Feng, X. Unleashing the low-precision computation potential of tensor cores on gpus. In IEEE/ACM Int. Symp. on Code Generation and Optimization (CGO), pp- 90-102, 2021. doi: 10. /000//CCOO05551.22...99999
Ignatov, A., Malivenko, G., Timofte, R., Treszezotko, L., Chang, X., Ksiazek, P., Lopuszynski, M., Pioro, M., Rud- nicki, R., Smyl, M., Ma, Y., Li, Z., Chen, Z.,, Xu, J., Liu,
Li, Z., Yang, T., Wang, P., and Cheng, J. Q-T:T: Fully differentiable quantization for vision transformer. arXiv, 2022. doi: (0.48550/arXiv.2200 .07703.
10

---

Predicting Iligh-precision Depth Low-Precision Devices Using 2D Hilbert Curves
Lin, T.-Y., Maire, M., Belongie, S. J., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L., Microsoft coco: Common objects in context. In Eur. Conf. Comput. 148.
Rasla, A. and Beyeler, M. The relative importance of depth cues and semantic edges for indoor mobility us- ing simulated prosthetic vision in immersive virtual re- ality. In Proc. of ACM Symp. on Virtual Reality Soft- ware and Tech., New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450398893. doi: 10.1 14453562939.35556222
Vis., 2014. doi:
Lipson, L.,, Teed, Z.,, and Deng. J. RAFT-stereo: Multilevel recurrent field transforms for stereo matching. In Int. Conf. on 3D Vision (3DV), pp. 218-227 2021. doi: 10. (109/3DV53792.22211. 100032.
Rouse, D. M. and Hemami, S. S. Understanding and sim- plifying the structural similarity metric. In IEEE Int. Conf. Image Process. pp. 1188-1191, 11 2008. doi: 10. 10.1 10.1109//1122000.4440000 471119
Liu, Q. and Zhou, S. LightDepthNet: Lightweight CNN architecture for monocular depth estimation on edge de- vices. IEEE Trans. on Circuits and Systems II: Express Briefs, 71(4):289-2233,, 2023. doi: 10.1109/TCS1.2023. 3337369.
Rousseeuw, P. J. and Croux, C. Alternatiess to the median absolute deviation. J. of the American Statistical Associa- rion, 88: 18::12731228 1283, 1993. doi: 10. 10.108000121459..11 10476408.
Mayer, N., 1lg. E., Häusser, P., Fischer, P., Cremers, D., Dosovitskiy, A., and Brox, T. A large dataset to train con- volutional networks for disparity, optical flow, and scene flow estimation. In IEEE Conf. Comput. Vis. Pattern Recog., June 2016. doi: 10. 00.1099/vpp.20164433.
Sagan, H. Space-filling curves. Springer Science Business Media, 2012.
Sakr, C., Dai, S., Venkatesan, R., Zimmer, B., Dally, W., and Khailany, B. Optimal clipping and magnitude- differentiation for improved quantization-awve In Int. Conf. Mach. Learn., volume 162, pp. 19138-19138. PMLR, 2022. URL https::/ press/ve /ols/vvlee 62/ sawr 2atr22asaakk222 .dff.
Nagel, M., Fournarakis, M., Amjad, R. A., Bondarenko, Y., Baalen, M. V., and Blankevoort, T. A white paper on neural network quantization. arXiv, 2021, doi: 10.48550/ arXiv.2106.08295.
training,
mmlr.
Nicolas, B., Li, Y., Chris, P., and Aaron, C. Delving deeper into convolutional networks for learning video represen- tations. arXiv, 2015. doi: 10.48550/arXiv.1551 .66432.
Shao, J., Yang, Y., Zhou, H., Zhang, Y., Shen, Y., Poggi, M.. and Liao, Y. Learning temporally consistent video depth from video diffusion priors. arXiv, 2024. doi: 10-48550/arxxv.2446001444
Oh, S., Kim, H.-. S., Lee, J., and Kim, J. RRNet: aopetition-reductiinn network for energy efficient depth estimation. IEEE Access, 8: 8:106097-106108 2020. doi:
Shao, S., Pei, Z., Chen, W., Wu, X., and Li, Z. NDDepth: Normal-distance assisted monocular depth estimation. In Int. Conf. Comput. Vis., pp. 7931-7940, 2023. doi: 10. 1099CCCV110700023.00000
Oquab, M., Darcet, T., Moutakanni, T., Vo, H. Q., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F, El-Nouby, A., Assran, M., Ballas, N., Galuba, W. Howes, R., Huang, P.-Y. B., Li, S.W Misra, I., Rab- bat, M. G., Sharma, V., Synnaeve, G., Xu, H.. Jégou, H., Mairal, J., Labatut, P., Joulin, A., and Bojanowski, P. DINOv2: Learning robust visual features without supervision. Trans. on Machine Learning Research, 2024. ISSN 2835-8856. doi: 10.48550/a2Xiv233044 07193. URL http:s///ppeeeviiieeeeee wttp:/iwwwweeeootooe id=a68sUt 6zFt.
Sheng, T., Feng, C., Zhuo, S., Zhang, X., Shen, L., and Aleksic, M. A quantization friendly separable convolu- tion for MobileNets. In /st Workshop on Energy Effi- cient Machine Learning and Cognitive Computing for Embedded Applications (EMC2), pp. 14-18, 3 2018. doi: 10. AA.1099EMC2.2211880000
Shkolnik, M., Chmiel, B., Banner, R., Shomron, G., Nahshan, Y., Bronstein, A., and Weiser. U. Robust quantization: one model to rule them all. In Adv. Neural Inform. Process. Syst., vol- ume 33, 2020. URL https::/prroeeett
Qualcomm Technologies, Inc Snapdragon neural processing engine SDK, 2025. URL https://
www www.couammmm.omm www//owweeeeeeeeeeooooocoeo.ww.e http///eeoooe//oowarrr.re heuraal-rrocessin---e sdd-
3 #45eao8..s2294422100008833333333 pdf.
Ranftl, R., Bochkovskiy, A., and Koltun, V. Vision trans- formers for dense prediction. In Int. Conf. Comput. Vis., pp. 12179-12188, October 2021. doi: 10.1 ICCV44992.2021.001199
Shlezinger, N., Eldar, Y. C., and Rodrigues, M. R. D. siddware-limited task-based quantization. IEEE

---

Predicting High- nigh-preccison Depth Low-Precision Devices Using 2D Hilbert Curves
Int. Workshop on Signal Process. Advances in Wire- less Communications (SPAWC), pp. 1-5, 8 2019. doi: AAL109/SPAW..211.8881555
Xiao, B., Wu, H., and Wei, Y. Simple baselines for human pose estimation and tracking. In Eur: Conf. Comput. Vis., pp. 472-487, 2018, doi: 10.
Song, L., Shi, D., Xia, J., Ouyang, Q. Qiao, Z., Jin, S., and Yang, S. -aate dynamic lightweight self- supervised monocular depth estimation. IEEE Robotics and Automation Lett., 9():8833890, 2023, doi: 10.1109/ L.3.2923.3337991.
Xu, G., Wang, X., Ding, X., and Yang. X. Iterative geometry encoding volume for stereo matching. In IEEE Conf. Comput. Vis. Puttern Recog-, pp. 21919-21928, 2023. doi: 10.1109/VVPS2229 201.R222299.0220000999.
Yang, J., Shen, X., Xing, J., Tian, X., Li, H., Deng, B., Huang, J., and sheng Hua, X. Quantization networks. In
Toshev, A. and Szegedy, C. DeepPose: Human pose esti- mation via deep neural networks. In IEEE Conf. Com- put. Vis. Pattern Recog., pp. 1653-1660, 2014. doi: N0.11099VVR.2204.214
IEEE Conf. Comput. Vis. Pattern Recog., pp. 7308-7316,
2019. doi: 10.1109/CVP.2201900077
Yang, L.,, Kang, B., Huang, Z., Xu, X., Feng, J., and Zhao, H. Depth anything: Unleashing the power of large-scale unlabeled data. IEEE Conf. Comput. Vis. Pattern Recog.. pp. 10371-10381, 2024a. doi: 10.1109/CVPRS2773. 2024.00987.
Ventrella, J. Brainfilling curves fractal bestiary. Eye- brain Books, 2012.
Wadekar, S. N. and Chaurasia, A. VobileViTv3: mobile- friendly vision transformer with simple and effective fu- sion of local, global and input features. arXiv, 2022. doi: 1448550/arXiv2220..5555
Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., and Zhao, H. Depth anything v2, arXiv, 2024b. doi: 10. 01.550/arXi..2406.04949
Wang, K., Liu, Z., Lin, Y., Lin, J., and Han, S. HAQ:
Yeshwanth, C., Liu, Y.-.., Nießner, M., and Dai, A. Scan- Net++: A high fideliiy dataset of 3D indoor scenes. In Int. Conf. Comput. Vis., pp. 12-22, 2023. doi: 10. (009/CCVV10002223..000
Hardware-awar awwre automated quantization with mixed pre- cision. In IEEE Conf. Comput. Vis. Pattern Recog, pp. 8612, 2019a. doi: 10.1199CVPR.2219.000088
Wang. T., Wang, K., Cai, H., Lin, J., Liu, Z., and Han, S. APQ: Joint search for network architecture, pruning and quantization policy. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. doi: 10.1 10110//VP22200000000000
You, Y., Wang, Y., Chao, W.-L., Garg, D., Pleiss, G., Hariha- ran, B., Campbell, M. E., and Weinberger, K. Q. Pseudo- LiDAR++: Accurate depth for 3D object detection in autonomous driving. In Inf. Conf. Learn. Represent., https://openrevier /oppnreveeeeeeeeeeeeeeeeeereeeee id-BJecHRVtPB.
Wang, Y., Chao, W.-L.., Garg, D.. Hariharan, B., Campbell, M., and Weinberger, K. Q. Pseudo-LiDAR from visual depth estimation: Bridging the gap in 3D object detection for autonomous driving. In IEEE Conf. Comput. Vis. Pattern Recog., pp. 8437-8445, 2019b. doi: 10.1109/ CVPR.2019.00864.
2020. URL https:
net forum?
Yuan, Z., Xue, C., Chen, Y., Wu, Q. and Sun, G. PTQ4VIT: Post-training quantization for vision trans- formers with twin uniform quantization. In Eur. Conf. Comput. Vis., pp. 191-207. Springer, 2022. doi: 10.1007/ 978-3-031-19775
Wang, Z., Bovik, A. C., and Sheikh, H. R. Structural sim- ilarity based image quality assessment. Digital Video Image Quality and Perceptual Coding, Ser. Series in Signal Process. and Communications, 11 2005. doi:
Zhang, N., Nex, F,, Vosselman, G., and Kerle, N. Channel- aware distillation transformer for depth estimation on nano drones. arXiv, 2023. doi: 10.48350/arxiv.2303. 10386.
Wofk, D., Ma, F, Yang, T.-.,, Karaman, S., and Sze, V. FesthD Fast monocular depth estimation on embed- ded systems. In Int. Conf. on Robotics and Automation (ICRA), pp. 6101-6108, 2019. doi: 10. 10.1109//....219. 8794182.
Zhou, Q.-Y,, Park, J., and Koltun, V. Open3D: A modern library for 3D data processing. arXiv, 2018. doi: 10. 48550/arXiv 1801 .098477
Xianqi, W., Gangwei, X., Hao, J., and Xin, Y. Selective- stereo: Adaptive frequency information selection for stereo matching. In JEEE Conf. Comput. Vis. Pat- tem Recog., pp. 19701-19710, 2024. doi: 10.1109/ CVPR52233.20024.08863
12

---

Predicting High-preeiiiin Depth Low-Precision Devices Using 2D Hilbert Curves
A. Parametric curve selection
The Hilbert curve is one representative of the wide class of space- filling curves (Ventrella, 2012). Let us provide additional arguments in favor Hilbert curve selection for high-precision depth prediction. In analysis, follow the terminology of (Ventrella, 2012) that classifies curves square and triangular grids, and divides them into families VN. In family VN, the distance between starting and ending points of the curve generator equals to VN.
In order to fully utilize depth representation as two components, it is desirable that space-filling curve uniformly covers the unit square. This requirement eliminates all curves on triangular grids.
Curves with non-orthogonal generators have the drawback of filling square not uniformly as for example Z-order curves. Among curves with orthogonal generation curves defined on square grid our choice limited to the Hilbert curve (v4 family) (Fig. 11a), the Peano curve (v9 (Fig. 1b) and the Quadratic Gosper curve (v25 (Fig. 11c.. For Hilbert and Peano curves different generators possible (e.g., the Moore curve is variant of the Hilbert curve) but they different only in the order the space is filled.
(b)
The value of N defines how fast the curve length Lp increases and curve edge size hy decreases with the curve order. Our experiments show that it is desirable to have the ability of fine-tuning the curve length depending on the DNN quantization error magnitude. From this point of view, the Hilbert curve is the most flexible as it has the lowest N value. For the Hilbert curve, the number of nodes grows 1, 4, 16, 64, 256 with the order p. For Peano curve, the number of nodes grows as 1, 9, 81, 729, 6561, and for Quadratic Gosper curve as 1, 25, 625, I 15625, 390625. If we limit the number of nodes to a reasonable value of 256, the Hilbert curve provides 4 usable low-order curves (that we experiment with the paper), Peano 2, Quadratic Gosper -
Figure 11. Space- filling curves filling unit square. The first (left column) and the second (right column) order curves for: (a) Hilbert curve, (b) Peano curve, and (c) Quadratic Gosper curve.
The Hilbert curve is the simplest and the most flexible curve that satisfies all requirements essential for coding depth values. To construct even more flexible list of curves, it is possible to use Hilbert, Peano and Quadratic Gosper curve of different orders to create sequence of curves with the number of nodes 4, 9, 16, 25, 64, 81, 256.
Provided the main requirements for the parametric curve (self-avoidance, uniform filling of unit square, continuity) are satisfied, the detailed structure of the curve is not important. For example, we can use arbitrary non-self-similar curves that fill unit square with a given number of nodes, curves with smoothed corners, curves that stretch different parts of ID value in different degree (to emphasize the most probable range of target quantity variation).
B. DPT model modification
As mentioned in the paper, one of the models chosen for the experiments is Dense Prediction Transformer (DPT) (Ranftl al., 2021). All modifications of the model architecture are illustrated in Fig. 12. An additional 2D convolution layer was used for proper integration of the input RGB stereo pair into the MibileViTv3.. (Wadekar Chaurasia, 2022) backbone.
Also, MobileNet blocks in the encoder are modified for better quantization described by Sheng al. (Sheng et al., 2018). For disparity prediction, the original DPT head was used. The Hilbert curve head architecture for this model includes an additional up-sample layer after the first 3 x 3 2D convolution layer.
The DPT model includes a VobileViTv3-S encoder with skip connections before each Mobile ViT block. Each skip
13

---

Predicting High-preeiiiin Depth Low-Precision Devices Using 2D Hilbert Curves
MobileViTv3 backbone
DPT decoder
Input
1x1 2D conv. 3 filters
Stereo pair
Skip connection
with reassemble block
1x1 2D conv. filters
3x3 2D conv. 32 filters
Hilbert Curve points
3x3 2D conv. 128 filters
Gaussian noise (optional)
Disparity map
LUT
1x1 2D conv. filters
3x3 2D conv. 32 filters
Hilbert Curve
Output
points
Proposed DNN modification
Figure 12. The hpDPT model architecture. The input RGB stereo pair is processed by an encoder which is MobileViTv3-S backbone and the decoder proposed for depth prediction in DPT. Features from the decoder are fed to optional Gaussian noise layer and 2D convolution layer followed by two heads for Hilbert curve components. They consist of one 3 x 3 and one x 2D convolution layers with a decreasing number of filters: and respectively. At the post-processing stage, Hilbert components are converted to the final disparity map.
connection integrates into the decoder part using reassemble block proposed by Ranftl et al. (Ranftl et al., 2021).
During analysis of the network architecture, we found that layers in MobileNet blocks have large kurtosis values of their weights' distributions. is suggested in (Shkolnik et al., 2020) that large kurtosis values might lead to the model quantization quality degradation because of outliers clipping. We added kurtosis regularization proposed (Shkolnik er al., 2020) to all 1 x convolutions MobileNet blocks in MobileViTv3-S reduce quantization error in both experiments with depth and Hilbert components output.
C. Experiment on KITTI 2012 dataset
We evaluate our approach on KITTI 2012 (Geiger et al., 2012) dataset. KITTI 2012 a real-world dataset for autonomous driving domain with sparse ground-truth disparities collected with a LiDAR sys- Our evaluation is based 194 images in the training part of KITTI 2012. Training dataset is composed of ScanNet and Virtual KITTI 2 (Cabon et al., 2020) datasets with 25/75% balancing. The DispNet and h2DispNet models were trained in 256 by 1 1152 px imput resolution and 128 by 576 px output disparity resolution. Training settings are the same as for the ScanNet experiment.
Precision
Ahs Rel, %
EPF, px 1
DI, J
FP32
3,88
1.38
9.13
1.63
Ours, FP32
3,24
1.07
5,22
Ours, W8A8
3.30
1.09
5.36
Table 3. Results on KITTI 2012 for the DispNet and h2DispNet models. Models FP32 format CPU; models W8A8 formats are run on DSP. Best metrics for W8A8 models shown in bold.
As shown in Table 3, for the original DispNet model we reached EPE 1.38 px and D1 9.13%. Interestingly, h2DispNet shows better results with EPE 1.07 px and D1 5.22%. We exclude S. from the analysis, because it is not applicable sparse ground-truth disparities of KITTI 2012 dataset. The DispNet model quantized to W8A8 format and run DSP shows degradation of both EPE (to 1.63 px) and DI (to 9.95%). At the same time the h2DispNet in W8A8 format on DSP shows very minor quality degradation. Measured as EPE between disparity predicted by FP32 and W8A8 models, quantization error is 1.01 px for the DispNet and 0.38 px for the h2DispNet. It corresponds to gain disparity prediction quality on device 2.6 times. Overall, on DSP the h2DispNet improves D1 by approximately 4.6% and EPE by about 33% compared to the original
14

---

Predicting Depth Low-Precision Devices Using 2D Hilbert Curves
(a) Image
(b) GT disparity
(c) DispNet disparity, FP32
(d) h2DispNet disparity, FP32
(c) DispNet disparity, W8A8
(f) NeDispNet disparity, W8A8
(g) |FP32- W8A8), DispNet
(h) |FP32 W8A8, h2DispNet
(i) (088), Disp
(i) GT W8A8), h2DispNet
Figure Comparison of DispNet and NeDispNet on KITTI 2012 dataset. Differences between disparity maps predicted by FP32 (CPU) (c, d) and W8A8 (DSP) (c. f) models shown in (g) and (h); absolute disparity prediction errors for W8A8 (DSP) models shown in (i) and (j). All values are presented in pixels.
DispNet model. Examples of predicted disparity are given in Fig. 13. This experiment demonstrates that the proposed approach can be successfully applied to real datasets.
D. Quantization quality influence on mesh fusion
We provide additional experiment to understand how quantization artifacts depth maps affect scene mesh reconstruction. For mesh fusion we utilize truncated signed distance function (TSDF) (Curless & Levoy, 1996) approach as implemented in Python library Open3I) (Zhou al., 2018). We used scalable TSDF volume with parameters vorel_lengie sdf trune 0.15. For experiments chose ScanNet scene scene0050. 02 comprising 4379 frames. Each 40th frame was used to reconstruct mesh. Quality of the mesh fused from GT depth maps is illustrated in Fig. 14.
0.01 m.
In Fig. 15 (Fig. 16), we show qualitative results of h2DispNNtt (h2DPT) model compared to the corresponding baseline variant. 3D meshes fused from depth maps predicted by FP32 h2DispNet (Fig. 15b) and FP32 h2DPT (Fig. 16b) models
have very similar structure and depth smoothness compared the FP32 DispNet (Fig. 15a) and FP32 DPT (Fig. 16a). Both baseline and modified FP32 models' variants produce quality of reconstructed 3D mesh comparable GT (Fig. 14) but with slightly smoother structure.
We observe two types of quantization artifacts present in fused meshes for models run on CPU delegate. The first is the noise on flat surfaces for original DispNet in W8A8 format (Fig. 15e); the second is presence of visible edges of different frames' depth maps (skep-like structures) in the mesh for original DPT W8A8 format (Fig. 16e). We attribute the second type of artifacts to the systematic errors in depth prediction leading to errors depth scale. Models modified according to the proposed method lead mesh reconstruction with much reduced noise level (Figs. 15f and 16f).
15

---

Predicting Ilig-pprecision Depth Low-Precision Devices Using 2D Hilbert Curves
(a) GT mesh with texture
(b) GT mesh without texture
Figure 14, A view of 3D mesh fused with GT depth maps for ScanNet scene scene0050 02.
For the baseline models run DSP delegate, observe the same artifacts but more pronounced (Figs. 15c-16c). The h2DispNet model in W8A8 format almost eliminates quantization artifacts (Fig. 15d) and restores mesh spatial details. The h2DPT in W8A8 format model removes step-like artifacts and reduces noise for flat surfaces.
E. Additional details of depth maps quality
In Figs. 17-21 we show additional examples depth maps predicted by original and modified DispNet and DPT models. These examples represent both simpler (Figs. 17-20) and more complex scenes (Fig. 21). In all examples, modified models have significantly smaller quantization error; remaining errors concentrated depth discontinuities. Errors in the vicinity of depth discontinuities are also present for FP32 models and not linked to the proposed approach.
F. Human pose estimation experiment
Two approaches were realized for the Human pose estimation (HPE) experiment: direct keypoints regression (Toshev & Szegedy, 2014) and via representing keypoints as heatmaps (Xiao al., 2018). For training, the MS COCO (Lin et al., 2014) dataset with human pose keypoints labeling was used. Both models have ResNer-RS (Bello al., 2021) an encoder with alpha equals 0.5. The head for direct regression is composed of fully-connected layers was introduced by Toshev et al. The model for heatmaps prediction includes DispNet decoder and convolutional head proposed by Xiao et al.
For HPE experiment we trained two lightweight models: direct keypoints prediction with AP = 43, and heatmaps prediction with AP30 83. We found that the most stable way to train the modified HPE model is training the original model first, using it as a teacher model, and calculating GT Hilbert curve components Ter and yor from the teacher model predictions the loss term (3). this manner, the modified model can be trained even when GT data for calculation of zar and yor
available, for example in case of heatmaps learning.
For model with direct keypoints prediction, dlong-the-curve quantization error for model with Hilbert curve-based represen- tation for p = 1, 2 increases approximately L times as compared to the keypoints quantization error of the original model (see Fig. 22a for p = 1 case); across-the-curve error is an order of magnitude smaller and is similar to the quantization error of the original model. As a result, quantization error of the modified model in keypoints space remains almost unchanged. This result suggests that for the direct regression task quantization error is strongly signal-dependent. The only effect from the proposed modification is in increase of output bit- preciseiiiion.
The situation is different for the heatmaps prediction model. In this case, quantization error along the Hilbert curve of the second order increases less significantly as compared to the original model. From Fig. 22b it is seen that the along-the-curve eиror distribution is narrower than the original heatmaps quantization error multiplied by the curve length. As a result, quantization error of the modified model in heatmaps space 2.69 times smaller than for the original model. In addition, the spatial structure of predicted heatmaps is improved as shown in Figs. 22c and 22d. The latter effect is caused by increase of the output bit- bit-precision.
16

---

Predicting Depth Low-Precision Devices Using 2D Hilbert Curves
(a) DispNet, FP32
(b) h2DispNet, FP32
(c) DispNet, WSA8DSS)
(d) h2DispNet, WSA8(DSP)
(e) DispNe, WSA8(CPU)
(I) h2DispNet, W8AS(CPU)
Figure 15. A view of 3D mesh fused with predicted depth maps by DispNet and h2DispNet for ScanNet scene scene0050_02. Some reconstruction emrors are highlighted by red and improved structures are marked by green.
17

---

Predicting Depth Low-Precision Devices Using 2D Hilbert Curves
DPT, FP32
(b) h2DPT, FP32
(c) DPT. WSA&DDSP
(d) h2DPT. WSA8(DSP)
(e) DPT, W8AS(CPU)
(I) h2DPT, WSA8(CPU)
Figure 16. A view of 3D mesh fused with predicted depth maps by DPT and h2DPT for ScanNet seene scene0050_02. Some reconstruction errrrs are highlighted by red and improved structures are marked by green.
18

---

Predicting High-precisiin Depth Low-Precision Devices Using 2D Hilbert Curves
3.0
2.5
1.5
1.0
0.5
(a) GT depth
(b) DPT depth
(c) h3DPT depth
0.5
0.4
0.3
0.2
0.1
(d) Image
(c)
DPT]
i GT h3DPT]
Figure Depth errors DPT and h3DPT models on DSP. ScanNet scene scene00330 02. All values presented in meters.
2.5
2.0
1.5
1.0
0.5
(a) GT depth
(b) DPT depth
(c) h3DPT depth
0.4
0.3
0.2
0.1
(d) Image
(e) GT DPT
O GT h3DPT
Figure 18. Depth emrors of DPT and h3DPT models DSP. ScanNet scene0629 00. All values presented in meters.
19

---

Predicting High-precising Depth Low-Precision Devices Using 2D Hilbert Curves
3.0
2.5
2.0
1.5
1.0
0.5
(a) GT depth
(b) DispNet depth
(c) h3DispNet depth
0.5
0.4
0.3
0.2
0.1
(d) Image
(e) GT DispNet
(f)
h3DispNetl
Figure 19. Depth errors DispNet and h3DispNet models DSP. ScanNet scene scene0030 02, All values are presented in meters.
2.5
2.0
1.5
1.0
0.5
(a) GT depth
(b) DispNet depth
(c) h3DispNet depth
0.4
0.3
0.2
0.1
(d) Image
(e) DispNet
(I hhDDsppNell
Figure 20. Depth emorr of DispNet and h3DispNet models DSP. ScanNet scene0629 00. All values presented in meters.
20

---

Predicting figh-precision Depth Low-Precision Devices Using 2D Hilbert Curves
3.5
3.0
2.5
2.0
1.5
1.0
0.5
(a) GT depth
(b) DPT depth
(c) h3DPT depth
R.K rx
0.4
0.3
0.2
0.1
(d) Image
(e) GT DPT]
(1 GT h3DPT
Figure 21. Depth errors DPT and h3DPT models on DSP, ScanNet scene scenc0804_00 All values presented in meters.
21

---

Predicting Depth Low-Precision Devices Using 2D Hilbert Curves
Original, multiplied by
Original, multiplied by
30
200
Along the curve
Along the curve
Probability density
25
Probability density
150
20
15
50
0
-0.2
0.1
0.0
0.1
0.2
-0.02
-0.01
0.00
0.01
0.02
Error, dimensionless
Error, dimensionless
(a) Keypoints error
(b) Heatmaps error
(c) Hcatmap by the original HPE model
(d) Heatmap by the modified HPE model
Figure 22. Quantization emorr for HPE using direct keypoints regression (a) and heatmaps (b). Comparison of heatmaps prediction on DSP using W8A8 models (c, d).
22


Reviews:
[{"review_id": "gZu5fWY2e4", "reviewer": "Reviewer_aLW4", "summary": "This paper focuses on high-precision depth estimation on low-precision devices. It leverages a 2D Hilbert Curves for better representation of high dynamic range depth. Extensive experimental results demonstrate the superiority of the proposed method on both accuracy and computational overhead.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer aLW4 for his/her positive feedback. We are encouraged that aLW4 found the paper well-argued and well-structured, recognized clarity of the main idea, effectiveness of the proposed method and soundness of the experimental analysis. We address reviewer comments below and will incorporate all feedback in the final version.\n- **“I suggest thickening the text in Figure 2 and the lines in Figure 4. They are too thin to be seen in the paper”**: Agree, we will make this change in the camera-ready paper.\n- **“The depth estimation results shown in the paper are predicted in scenes with simple structures. Can you show the depth estimation results of some complex scenes?”**: Yes, we can add an example for a more complex scene. We selected a complex frame representing a kitchen with many small objects on the kitchen table and kitchen cabinet with many shelves (ScanNet, scene0804_00, camera pose for frame #340). This example confirms quantization error reduction after applying our approach, and we will add it in the Appendix D of the camera-ready paper.", "claims_and_evidence": "Yes, the paper clearly states the reasons for introducing the Hilbert curve and gives a proper analysis.", "methods_and_evaluation_criteria": "Yes, the evaluation includes estimation accuracy, runtime, and power consumption, which are appropriate for the application at hand.", "theoretical_claims": "The paper provides a proper analysis of the expected properties of the parametric curves and loss function modifications.", "experimental_designs_or_analyses": "I have checked the experimental results in the main text.", "relation_to_broader_scientific_literature": "This paper targets on the depth estimation problem and the evaluation is based on two prior depth estimation works: DispNet and DPT.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "- The paper is well-structured and well-argued.\n- The experimental results demonstrate the effectiveness of proposed method.", "other_comments_or_suggestions": "- I suggest thickening the text in Figure 2 and the lines in Figure 4. They are too thin to be seen in the paper.", "questions_for_authors": "- The depth estimation results shown in the paper are predicted in scenes with simple structures. Can you show the depth estimation results of some complex scenes?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SmCGGpmiaZ", "reviewer": "Reviewer_GYWs", "summary": "The paper presents a method for achieving high-precision depth prediction on low-precision devices by representing depth as two components of a 2D Hilbert curve. A full-precision DNN is trained to predict these components, and a post-processing step reconstructs high-precision depth from low-precision predictions. The key findings indicate that the method can increase the bit precision of predicted depth by up to three bits and reduce quantization error by up to 4.6 times. Experiments show that the modified model, quantized to the W8A8 format, can achieve similar or better depth prediction quality than the original W8A16 model, with reduced inference time and power consumption.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer GYWs for his/her positive feedback. We are glad GYWs has found our idea innovative and recognized that we provided substantial evidence to validate our claims. We are encouraged that GYWs found our approach novel, with high practical value and broad application prospects. We address reviewer comments below and will incorporate all feedback in the final version.\n- **“… the assumption of independent quantization errors for Hilbert components does not fully hold in practice, which may affect the universality of some conclusions....”; “… more attention should be paid to the impact of the discrepancy between actual data and theoretical assumptions on the results when analyzing quantization errors”; “The assumption of independent quantization errors for Hilbert components does not match actual data, potentially affecting the universality of the conclusions”**: We believe that some misunderstanding may stem from the statement “Quantization error compression takes place if (a) Hilbert components quantization error is independent between channels and identically distributed …” in Subsection 2.4., L212. Our main goal was to overcome the bit-precision limitation of low-end devices, and this goal is achieved regardless of the independence of quantization errors. We observe the bit-precision increase in all experiments (depth prediction and human pose estimation) and for all models (DispNet, DPT, ResNet-RS). In addition to this main goal, we assumed that quantization error after transforming Hilbert components to the depth/disparity map can be reduced by up to the Hilbert curve length times, however, this effect requires independency of quantization error in Hilbert components. If errors in Hilbert components are dependent, only the bit-precision increases, if they are independent, the bit-precision increases and quantization error reduces. Our experiments revealed that this effect does take place, but the independence of Hilbert components is task-dependent. We consider this result not as a limitation of the proposed idea but as an indication of its potential beyond our initial goal. Being limited by the paper size, we focused on explaining the main idea. In addition, we made our best efforts to describe the observed effect of quantization error reduction and provide our working hypothesis supported by available experiments: the effect strength depends on relation between quantization errors of Hilbert curve components. We hope that further experiments can clarify this question, probably, by assuring quantization errors independence in the loss function with quantization errors modelling with QAT. In the camera-ready paper, we will stress that assumption of independent quantization errors for Hilbert components is not needed for achieving our main goal of increasing bit-precision.\n- **“Since no information about the supplementary material is available”**: According to the ICML call for papers, supplementary materials should not be submitted as a separate file but rather included as appendices after the main paper body. In our paper, supplementary materials are present as appendices A-E (pages 12-19), where we provide additional information on space-filling curve selection, DPT model modification, experiment on mesh fusion quality, more examples of predicted depth maps, and experiments on human pose estimation. Each appendix has a reference in the main paper body. We hope this additional information is available and will provide valuable details of our work.\n- **“…the differences and advantages of this method compared to other methods for improving quantization errors could be further emphasized.”** To address this suggestion, we propose to add a short subsection in the camera-ready paper with the following content: (1) an illustration of the required modification of a full-precision model and its training pipeline, a block diagram of modified model inference on device with indication of parts that are run on DSP (Hilbert components prediction) and on CPU (post-processing), and data transfer between DSP and CPU; (2) additional text where we will emphasize that existing quantization methods (PTQ, QAT) can be integrated into this scheme and bring improvement in specific domains or for specific architectures. We will also emphasize the difference between PTQ, QAT, and our method.\n- **“The effectiveness of the method depends on specific hardware environments”** This is not quite so. Specific hardware influences all methods related to on-device models deployment because of supported arithmetics limitations. Our method adds a computationally simple post-processing step (LUT table) and can be applied to any hardware that supports quantized models inference. We experimented with CPU inference, DSP inference of FP16 model, DSP inference of W8A16 and W8A8 models and in all cases found improvement in quantization quality (higher bit-precision, lower standard deviation of quantization error).", "claims_and_evidence": "- Claims: The proposed method can improve the bit precision of predicted depth, reduce quantization error, and enable efficient depth prediction on low-precision devices.\n- Evidence: Experiments on DispNet and DPT models provide substantial evidence. By comparing the performance of the original and modified models in terms of metrics such as Abs Rel, RMSE, δ₁, and SC, as well as their performance on CPU and DSP, the paper validates these claims. However, the assumption of independent quantization errors for Hilbert components does not fully hold in practice, which may affect the universality of some conclusions, although the overall advantages of the method are still demonstrated.", "methods_and_evaluation_criteria": "- Methods: Representing depth as Hilbert curve components is innovative, and experiments verify its effectiveness in enhancing precision and reducing errors. Nevertheless, the method requires retraining the full-precision model and imposes certain constraints on the errors of the quantized depth prediction model, which may limit its practical applications.\n- Evaluation Criteria: Using standard depth prediction metrics (Abs Rel, RMSE, δ₁) and the newly proposed cosine similarity (SC) based on DCT coefficients to evaluate depth prediction quality is reasonable and comprehensive. Measuring quantization error using standard deviation (SD) is also appropriate for effectively assessing model performance.\n- Theoretical Claims：The theoretical analysis of using the Hilbert curve for depth coding in the paper is reasonable. The discussion on the properties of the curve (continuity, non-self-intersection, boundedness, and self-avoidance) is sufficient, explaining the rationale for choosing the Hilbert curve. The theoretical derivation in the quantization error analysis is logically sound. However, the discrepancy between the assumption of independent quantization errors of Hilbert components and the actual data in practical applications suggests a certain disconnect between theory and practice, which requires further research and improvement.", "theoretical_claims": "The theoretical analysis of using the Hilbert curve for depth coding in the paper is reasonable. The discussion on the properties of the curve (continuity, non-self-intersection, boundedness, and self-avoidance) is sufficient, explaining the rationale for choosing the Hilbert curve. The theoretical derivation in the quantization error analysis is logically sound. However, the discrepancy between the assumption of independent quantization errors of Hilbert components and the actual data in practical applications suggests a certain disconnect between theory and practice, which requires further research and improvement.", "experimental_designs_or_analyses": "- Experimental Designs: The selection of DispNet and DPT models for experiments is appropriate, and the modifications to the models are clearly described. The choice and processing of the dataset are reasonable, and the experimental settings (such as model input and output sizes, training-validation-test set division) are well-defined. Considering different models, quantization formats, and device operations (CPU and DSP) makes the experimental design comprehensive.\n- Experimental Analyses: The analysis of experimental results is detailed. By comparing the performance of the original and modified models across multiple indicators, evaluating the models from different perspectives (accuracy, error, runtime, power consumption), and conducting an in-depth analysis of quantization error reduction, the paper provides a comprehensive understanding. However, more attention should be paid to the impact of the discrepancy between actual data and theoretical assumptions on the results when analyzing quantization errors.", "relation_to_broader_scientific_literature": "The paper reviews relevant research in the field of depth prediction in the introduction, elaborating on the development trends and existing problems of current depth prediction methods. It clearly states that its method aims to address the issues of depth prediction accuracy and quantization error on low-precision devices, demonstrating a close relationship with previous studies. In the methodology section, relevant concepts such as quantization techniques and space-filling curves are cited and discussed, indicating both continuity and innovation in the research. However, when elaborating on the relationship with existing literature, the differences and advantages of this method compared to other methods for improving quantization errors could be further emphasized.", "essential_references_not_discussed": "none", "other_strengths_and_weaknesses": "Strengths:\n- Innovation: Representing depth using 2D Hilbert curves is novel, offering a new approach to depth prediction on low-precision devices.\n- Performance Improvement: Experimental results demonstrate that the method can effectively enhance depth prediction accuracy, reduce quantization errors, and decrease the inference time and power consumption of the model, showing high practical value.\n- Application Potential: The method is applicable to various depth prediction tasks, including monocular and binocular depth prediction, multi-view stereo, depth completion, depth quality enhancement, and depth inpainting, indicating broad application prospects.\n\nWeaknesses:\n- Model Training Cost: Retraining the full-precision model increases training costs and time.\n- Error Assumption Issue: The assumption of independent quantization errors for Hilbert components does not match actual data, potentially affecting the universality of the conclusions.\n- Hardware Dependence: The effectiveness of the method depends on specific hardware environments (such as devices supporting low-precision calculations), and it may not be applicable on some hardware that does not support such calculations.", "other_comments_or_suggestions": "none", "questions_for_authors": "none", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SFgEQCxWR3", "reviewer": "Reviewer_RnCh", "summary": "This paper proposes a new method for high-precision depth prediction on devices with low-precision arithmetic. The authors introduce an innovative technique that represents high dynamic range depth as two low dynamic range components of a 2D Hilbert curve. This approach enables depth maps with higher bit precision than conventional quantized 8-bit models. The algorithmic framework encompasses encoding and decoding depth information using 2D Hilbert curves, as well as incorporating this methodology into neural network training. Experiments with DispNet and DPT models on stereo matching tasks demonstrate the effectiveness of the proposed method.\n\n## update after rebuttal\n\nThank you for your valuable feedback! Major concerns regarding evaluation metrics, baseline comparisons, and figures have been addressed. \nAfter considering your feedback, I will increase my rating.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer RnCh for his/her thoughtful feedback. We are encouraged that RnCh found our claims supported by theoretical analysis and empirical results. We address reviewer comments below and will incorporate all feedback in the final version.\n- **“The evaluation metrics used are standard metrics for monocular depth estimation, rather than stereo matching (which would typically include EPE, D1, and D2)…”**: We agree that EPE and D1 metrics are typically used in the stereo matching domain. At the time of paper submission, we had both sets of metrics, including EPE and D1. Given the paper size constraint, we decided to include relative depth error and Sc (DCT cosine similarity) as primary metrics because they clearly reveal the effect of our idea application. In particular, Sc measures quality loss due to a decrease in bit-precision. For EPE and D1 metrics we have the following results. For the DispNet model, EPE/D1 is 0.29 px./1.81% for the full precision model and 0.69 px./5.34% for the W8A8 model; for h3DispNet EPE/D1 is 0.24 px./1.24% for the full precision model and 0.24 px./1.25% for the W8A8 model. The D2 metric is not applicable in our case, because we consider models that predict disparity only for the left frame. In the final paper version, we will update Table 1 and Table 2 by adding EPE and D1 metrics.\n - **“The selected baseline algorithms are insufficient to represent state-of-the-art techniques. For stereo matching, they could have included IGEV (CVPR'23), while for monocular depth estimation, NDDepth (ICCV'23) would have been appropriate comparisons”**: IGEV belongs to the class of stereo-matching models (RAFT-Stereo, Selective-IGEV) with recurrent refinement of predicted disparity using GRU or similar recurrent units. In this architecture, the predicted disparity is refined over multiple iterations by adding a small correction signal to the initially predicted disparity map. The integration of the proposed idea of depth/disparity representation as two Hilbert curve components into IGEV differs significantly from one-stage models like DispNet and DPT that we consider in the paper. While we propose to modify the output signal representation, IGEV modification requires integration of Hilbert components inside the model at multiple places. We think that our idea can be applied to IGEV, and this is currently our active research direction. However, this work is not completed, and even if completed, it cannot be added to the current publication without sacrificing clarity. A similar situation is with NDDepth: this model has depth refinement using GRU units. Apart from this, NDDepth predicts other components like normals, uncertainly maps, and planar regions that we do not claim to support at this stage of our research. In the camera-ready paper, we propose to clarify this situation by adding one paragraph to Discussion & Limitations section explaining that support of recurrent models requires further research efforts.\n- **“The paper lacks comparative experiments with other recent quantization methods...”** We would like to stress that we do not propose a new quantization method. Our idea is to modify a full precision model, use *existing* quantization methods without any modifications, and add a simple post-processing stage of quantized model output. Improved quantization quality provided by recent quantization methods (post-training quantization or quantization-aware training) will improve the quality of Hilbert components quantization and quality of depth/disparity calculated after post-processing. For a fair comparison, we need to find a similar approach that builds on top of existing quantization methods. However, in the available literature we found none. Therefore, we provided the best possible experimental evidence by comparing a standard PTQ quantization method (widely used SNPE library that quantizes models for Qualcomm chipsets) without and with our approach.\n- **“Why does Fig. 3(f) appear significantly different from the other visualizations. The visual discrepancy requires clarification to understand the results being presented properly.”**: The pair of images 3e, 3f shows decomposition of 16-bit depth into two 8-bit values, with 3e being the most significant byte and 3f – the least significant byte. In this representation, the least significant byte shows high-frequency variations making it significantly different in appearance from the original depth. We added these images to illustrate a possible but inconvenient way of factorizing high-precision depth into two low-precision components. To clarify this, in the camera-ready paper, we will add the following phrase in the Figure 3 caption “Fine details in (f) are the least significant byte of depth (a) represented in 16 bit format. High-frequency oscillations make it appear different from the original depth and difficult to predict by a DNN model”.", "claims_and_evidence": "1) The Hilbert curve method increases bit precision of predicted depth by up to three bits: This claim is supported by theoretical analysis and empirical results.\n2) W8A8 models with the proposed method outperform original W8A16 models while improving efficiency: This claim is supported by the experiments.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria are not well-aligned with the problem of enabling high-precision depth prediction on low-precision devices. A significant concern is that while the algorithm in this paper is designed for stereo matching, the evaluation metrics presented focus primarily on monocular depth estimation. The authors should supplement their analysis with standard stereo matching evaluation metrics to properly assess the method's performance in its intended application domain.", "theoretical_claims": "N.A.", "experimental_designs_or_analyses": "My major confusion regarding this paper stems from the experimental design.\n\n1) The evaluation metrics used are standard metrics for monocular depth estimation, rather than stereo matching (which would typically include EPE, D1, and D2). This creates a fundamental mismatch between the claimed application and its evaluation.\n2) The selected baseline algorithms are insufficient to represent state-of-the-art techniques. For stereo matching, they could have included IGEV (CVPR'23), while for monocular depth estimation, NDDepth (ICCV'23) would have been appropriate comparisons.\n3) The paper lacks comparative experiments with other recent quantization methods, which would be essential to properly contextualize the performance of the proposed approach.", "relation_to_broader_scientific_literature": "N.A", "essential_references_not_discussed": "N.A", "other_strengths_and_weaknesses": "N.A", "other_comments_or_suggestions": "1) Why does Fig. 3(f) appear significantly different from the other visualizations. The visual discrepancy requires clarification to understand the results being presented properly.", "questions_for_authors": "N.A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P2lfbZ0NAm", "reviewer": "Reviewer_QXvZ", "summary": "The paper presents an approach for neural network quantisation in monocular and binocular settings. The main idea is to decompose the high dynamic range depth into two low dynamic range components using a Hilbert curve and train a full precision DNN to predict these components. In practice, standard quantisation methods are used, followed by a post-processing step to reconstruct the depth from the low bit-accuracy Hilbert curve components. The performance of the approach is evaluated by testing the performance of depth estimation after quantisation of weights and activations; the models show superior performance despite lower quantisation levels using the proposed approach.\n\n\n## update after rebuttal\n\nHaving read all the reviews and replies, I maintain my positive score. The rebuttal provided important answers to the open points.  The new material should be included in the final version.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer QXvZ for his/her positive feedback. We are encouraged that QXvZ found the paper well-written and easy to follow, recognized novelty of the main idea, and soundness of the experimental design. We address reviewer comments below and will incorporate all feedback in the final version.\n- **“The motivation for using Hilbert curves needs further motivation. For example, one could explore alternative techniques such as Singular Value Decomposition (SVD) or Principal Component Analysis (PCA) to reduce the dimensionality of depth data and then reconstruct. This is a minor point, but it would help the paper.”**: SVD and PCA are methods for dimensionality reduction. In the context of our work, they are not applicable because depth has only one dimension. In contrast, we apply a transform that increases data dimensionality (single channel disparity/depth is transformed into two components of the Hilbert curve). In this manner, we transform high dynamic range depth into several lower dynamic range Hilbert curve components. Such a transformation can be done in many ways. We argued that not all of them are suitable for a full precision model training (e.g. 16 bit depth representation as the eight bit components is unsuitable, Figure 3, L141-151). We also provided arguments in favor of space-filling curves (L153-196). In addition, in the supplementary material, we discussed in detail why the Hilbert curve was specifically chosen among many available space-filling curves (Appendix A). Our intuition is that using other curves (e.g. Peano and Quadratic Gosper) is possible but can provide benefits only in some specific applications. Therefore, for the paper clarity and conciseness, we decided to concentrate on Hilbert curves.\n- **“The paper could discuss further works on monocular depth estimation with limited resources.”**: Thank you for suggesting this additional discussion and useful links! In the camera-ready paper, we will change the initial phrase in the first paragraph at page 2, Line 059 to “The efficiency of DNN inference on low-end devices can be achieved by applying several strategies. One is to optimize a model architecture to reduce number of parameters, latency, avoid using remove computationally intensive layers, use network pruning (Fast monocular depth estimation on embedded systems (2019), RRNet: Repetition- reduction network for energy efficient depth estimation (2020), LightDepthNet: Lightweight CNN architecture for monocular depth estimation on edge devices (2023), Spatial-aware dynamic lightweight self-supervised monocular depth estimation (2023)). Another strategy consists in usage of low-precision computations (Li et al., 2021; Jacob et al., 2018).”.\n- **“The results shown in Table 1 are not really easy to follow”**: Thank you for pointing this out! In the camera-ready paper, we will apply color coding of values for FP32 model, W8A8 model on DSP, W8A8 model on CPU (similar to the Table 2 at page 7) and explain it in the Table 1 caption. Also, we consider enlarging Table 1 from one to two columns because camera-ready paper provides one extra page for the main paper.\n- **“All experiments are performed on the ScanNet benchmark. More benchmarks would be welcome to support the claims made in the paper. The method should work equally well, but it needs to be demonstrated on other datasets, e.g. KITTI or NYU Depth V2.”** The NYU Depth V2 and ScanNet datasets are similar (indoor datasets collected by Kinect device). Our choice of ScanNet is determined by its size, availability of camera poses and mesh for each scene. During experiments design we seek how to check performance on other datasets and in other domains. Instead of the KITTI dataset suggested by the reviewer, which belongs to the same depth estimation domain, we selected the MS COCO dataset and Human pose estimation domain. Our results for the COCO dataset (Appendix E) support claims made in the paper. We don’t have results on the KITTI dataset at hand and time is needed to prepare them (to support KITTI in our own training framework, train full precision models, prepare a quantization dataset for the KITTI dataset, quantize models, and perform measurements on device). If this additional experiment is required, we are ready to perform it and report our findings in the final paper version. In short term (by 8th April), we can benchmark existing models trained on ScanNet on the training part of the KITTI dataset. In this case, we cannot guarantee competitive full precision models quality on KITTI, only compare full precision, FP16, W8A8, W8A16 models performance.", "claims_and_evidence": "The problem has often been approached using ML methods. The proposed methodology is well suited to the problem. Moreover, the evaluation is done on standard datasets.", "methods_and_evaluation_criteria": "Further evaluation at KITTI or NYU Depth V2 is recommended.", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "The experimental design is fine. There are missing evaluations.", "relation_to_broader_scientific_literature": "The output representation is novel compared to the current literature. The paper presented interesting ideas.", "essential_references_not_discussed": "The paper could discuss further works on monocular depth estimation with limited resources. For example:\n\n- Fast monocular depth estimation on embedded systems (2019).\n- RRNet: Repetition- reduction network for energy efficient depth estimation (2020).\n- Visual domain adaptation for monocular depth estimation on resource-constrained hardware (2021).\n- LightDepthNet: Lightweight CNN architecture for monocular depth estimation on edge devices (2023).\n- Spatial-aware dynamic lightweight self-supervised monocular depth estimation (2023).", "other_strengths_and_weaknesses": "Paper thicknesses: \n\n- The idea of predicting a point on a 2D parametric lower-order Hilbert curve is interesting. The paper presents a novel parametrization of the output space. \n- The paper is well written and easy to follow. The method is well described. \n- The approach works well in practice. The experiments show consistent and significant improvements over the standard quantisation protocol.\n\n\nWeaknesses of the paper:\n\n- The motivation for using Hilbert curves needs further motivation. For example, one could explore alternative techniques such as Singular Value Decomposition (SVD) or Principal Component Analysis (PCA) to reduce the dimensionality of depth data and then reconstruct. This is a minor point, but it would help the paper. \n- All experiments are performed on the ScanNet benchmark. More benchmarks would be welcome to support the claims made in the paper. The method should work equally well, but it needs to be demonstrated on other datasets, e.g. KITTI or NYU Depth V2.\n- The results shown in Table 1 are not really easy to follow, it is hard for the reader to know what exactly is being shown here, even after reading the text. The presentation of the results is crucial and clarity needs further work.", "other_comments_or_suggestions": "Not in particular.", "questions_for_authors": "A clearer method motivation would be helpful.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The authors present a method for high-precision depth estimation for low-precision arithmetic devices. The main contribution comes from modeling high dynamic range depth as two low dynamic range components of a 2D Hilbert curve. A neural network is trained to predict these components, which are used to reconstruct high-precision depth. Initial concerns on experiments and comparisons were addressed by the rebuttal. The reviewers agreed that the use of Hilbert curve for encoding depth is a sound choice and appreciate the discussion of its properties to support the rationale for choosing the Hilbert curve.
Decision: Accept (poster)