# AdaLite: A Distilled AdaBins Model for Depth Estimation on Resource-Limited Devices

Mohammed Chaouki Ziara, Mohamed Elbahri, Nasreddine Taleb, Kidiyo

Kpalma, Sid Ahmed El Mehdi Ardjoun

► To cite this version:

Mohammed Chaouki Ziara, Mohamed Elbahri, Nasreddine Taleb, Kidiyo Kpalma, Sid Ahmed El Mehdi Ardjoun. AdaLite: A Distilled AdaBins Model for Depth Estimation on Resource-Limited Devices. AI, 2025, 6 (11), pp.298. 10.3390/ai6110298 . hal-05395497

HAL Id: hal-05395497

https://hal.science/hal-05395497v1

Submitted on 12 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# AdaLite: A Distilled AdaBins Model for Depth Estimation on Resource-Limited Devices

Mohammed Chaouki Ziara<fnref n="1" /> ID, Mohamed Elbahri<fnref n="2" /> ID, Nasreddine Taleb<fnref n="1" /> ID, Kidiyo Kpalma<fnref n="3" /> ID
and Sid Ahmed El Mehdi Ardjoun<fnref n="4" />,* ID

## Abstract

This paper presents AdaLite, a knowledge distillation framework for monocular depth estimation designed for efficient deployment on resource-limited devices, without relying on quantization or pruning. While large-scale depth estimation networks achieve high accuracy, their computational and memory demands hinder real-time use. To address this problem, a large model is adopted as a teacher, and a compact encoder-decoder student with few trainable parameters is trained under a dual-supervision scheme that aligns its predictions with both teacher feature maps and ground-truth depths. AdaLite is evaluated on the NYUv2, SUN-RGBD and KITTI benchmarks using standard depth metrics and deployment-oriented measures, including inference latency. The distilled model achieves a 94% reduction in size and reaches 1.02 FPS on a Raspberry Pi 2 (2 GB CPU), while preserving 96.8% of the teacher's accuracy ($\delta_1$) and providing over 11x faster inference. These results demonstrate the effectiveness of distillation-driven compression for real-time depth estimation in resource-limited environments. The code is publicly available.

**Keywords:** depth estimation; resource-limited devices; distillation

## 1. Introduction

Monocular depth estimation (MDE) refers to the task of predicting scene depth from a single RGB image (see Figure 1 for an illustration). It constitutes a longstanding problem in computer vision with relevance to robotics, autonomous navigation, augmented reality, and 3D reconstruction [1,2].

<figure><img src="image_2.png" /><figcaption>**Figure 1.** Illustration of the depth and error maps from the proposed model: (Left to right) RGB image, predicted depth map, ground-truth depth map, and error map (dark indicates low prediction error, while brighter regions highlight areas of higher discrepancy). Inference time: 7.6 ms on GPU and 107 ms on CPU.</figcaption></figure>

<figure><img src="image_3.png" /></figure>

### ✓ check for updates

Academic Editor: Gianni D'Angelo

Received: 7 October 2025

Revised: 3 November 2025

Accepted: 11 November 2025

Published: 20 November 2025

**Citation:** Ziara, M.C.; Elbahri, M.; Taleb, N.; Kpalma, K.; Ardjoun, S.A.E.M. AdaLite: A Distilled AdaBins Model for Depth Estimation on Resource-Limited Devices. *AI 2025, 6, 298*. https://doi.org/10.3390/ai6110298

**Copyright:** © 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

<footnote n="1">¹ RCAM Laboratory, Department of Electronics, Djillali Liabes University, Sidi Bel Abbes 22000, Algeria; chaouki.ziara@univ-sba.dz (M.C.Z.); ne_taleb@univ-sba.dz (N.T.)</footnote>

<footnote n="2">² RCAM Laboratory, Department of Computer Science, Djillali Liabes University, Sidi Bel Abbes 22000, Algeria; mohamed.elbahri@univ-sba.dz</footnote>

<footnote n="3">³ Univ Rennes, INSA Rennes, CNRS, IETR-UMR 6164, F-35000 Rennes, France; kidiyo.kpalma@insa-rennes.fr</footnote>

<footnote n="4">⁴ IRECOM Laboratory, Djillali Liabes University, Sidi Bel Abbes 22000, Algeria</footnote>

<footnote>* Correspondence: elmehdi.ardjoun@univ-sba.dz</footnote>
Unlike stereo or multi-view approaches that exploit geometric correspondences, MDE relies exclusively on monocular cues, rendering the problem inherently ill-posed. Early deep learning methods approached MDE as a regression task, where depth values were predicted continuously at the pixel level under supervision from regression losses such as Manhattan distance (L1), Euclidean distance (L2), or scale-invariant formulations [3]. Although conceptually simple, regression-based approaches frequently suffered from over-smoothing, limited accuracy in regions of large depth variation, and the loss of fine structural detail [3]. To alleviate these limitations, Fu et al. [4] and Cao et al. [5] reformulated the problem as classification. In this paradigm, the depth range is discretized into bins and each pixel is assigned to the most probable bin. This design improves training stability and quantitative accuracy. However, discretization introduces visual artifacts such as discontinuities at bin boundaries, which compromise the perceptual quality of predicted depth maps. Hybrid regression-classification schemes were subsequently introduced to balance these trade-offs. Such methods estimate a probability distribution over bins and refine the prediction with a continuous regression term, combining the robustness of classification with the smoothness of regression. A further development is dynamic binning, where bin centers are predicted adaptively for each image, thereby improving sensitivity to local depth variations and enabling more reliable estimation across scenes with extreme depth ranges.

These formulations underpin several recent state-of-the-art models [6–10], which demonstrate consistent gains in accuracy, robustness, and cross-dataset generalization. Despite these advances, the majority of MDE networks remain unsuitable for deployment in production systems. This reflects a broader pattern in deep learning, where models achieving state-of-the-art accuracy often prove impractical for real-world use due to practical pitfalls [11]. The main limitations include high computational complexity, large memory requirements, reliance on specialized hardware, and fragile performance under distribution shifts. Such factors hinder adoption in real-time and resource-constrained applications. A central obstacle is the size of the model. Contemporary MDE architectures employ increasingly deep backbones and attention-based components, leading to high parameter counts and substantial floating-point operation (FLOP) costs. This complexity restricts deployment on embedded and mobile platforms and complicates integration into robotics and autonomous systems, where efficiency and reliability are essential. Figure 2 summarizes the approximate number of parameters and FLOPs for the representative models discussed above. FLOPs are reported for two common input resolutions, 224 × 224 and 640 × 480.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Model complexity of representative monocular depth estimation approaches. The results underscore the significant computational burden of current methods and motivate ongoing research toward compact and efficient architectures.</figcaption></figure>
Pruning and quantization are compression techniques that have been explored to enable models to perform real-time prediction on low-computing units. Pruning removes redundant and low-impact weights in the model using a random or criteria-based approach with a specified percentage, creating a sparse model and thereby reducing the total number of operations [12]. Meanwhile, quantization lowers the precision of model parameters from full precision (FP32) to half precision (FP16) or even integer levels (INT8 or UINT8) [13]. These strategies reduce model size and improve inference speed, but they can often introduce some accuracy degradation [14]. Table 1 illustrates the AdaBins model with its baseline results. We apply both pruning and quantization with different configurations and we evaluate the resulting trade-offs between accuracy and efficiency to highlight the practical limitations of current approaches.

To address these limitations, we propose leveraging recent advances in knowledge distillation for depth estimation, rather than compressing large models using conventional techniques or designing new complex lightweight architectures. The overall idea is to distill knowledge from a high-capacity teacher model to a lightweight student model that can operate in real time and under memory constraints. We train the student network with dual supervision: the student is optimized both from ground-truth depth supervision and from intermediate outputs of the teacher model. This formulation allows the student to capture fine-grained structural cues while remaining compact for deployment. Motivated by the success of knowledge distillation in tasks from image classification to semantic segmentation [15–17], we extend it to depth estimation, contributing the following:

1. Efficient distillation adaptation to dense prediction tasks;

2. A dual-supervision approach that capitalizes on both data and model guidance;

3. A lightweight depth estimation network that bridges the gap between research-level accuracy and deployable efficiency.

<table><caption>**Table 1.** Performance metrics of the AdaBins model on NYUv2 (Eigen Split [3]) at 480 × 640 resolution. Experiments were run on a CPU (Intel Core i7-10700, 8 cores, 16 threads) with 64 GB RAM. The baseline model was loaded from the publicly available repository (https://github.com/shariqfarooq123/AdaBins/ (accessed on 10 November 2025)) and run locally, yielding results consistent with the original paper [18]. The baseline model was subsequently pruned and quantized, and evaluated with the same code. Metrics reported include prediction accuracy, model size (MB), and inference speed (FPS). The baseline model is in gray. Arrows show metric direction, in the top row, ↑ means higher is better; in the bottom row, ↓ means lower is better.</caption><thead><tr><th>Model</th><th>δ<sub>1</sub> ↑</th><th>δ<sub>2</sub> ↑</th><th>δ<sub>3</sub> ↑</th><th>AbsRel ↓</th><th>SqRel ↓</th><th>RMS ↓</th><th>log10 ↓</th><th>Model Size ↓</th><th>FPS ↑</th></tr></thead><tbody><tr><td>AdaBins (Baseline)</td><td>0.902</td><td>0.983</td><td>0.997</td><td>0.103</td><td>0.057</td><td>0.369</td><td>0.044</td><td>299 MB</td><td>1.2</td></tr><tr><td>AdaBins (Pruned Magnitude 10%)</td><td>0.858</td><td>0.971</td><td>0.993</td><td>0.118</td><td>0.074</td><td>0.428</td><td>0.057</td><td>268 MB</td><td>1.5</td></tr><tr><td>AdaBins (Pruned Magnitude 30%)</td><td>0.713</td><td>0.923</td><td>0.981</td><td>0.160</td><td>0.127</td><td>0.605</td><td>0.079</td><td>210 MB</td><td>2.17</td></tr><tr><td>AdaBins (Pruned Magnitude 50%)</td><td>0.214</td><td>0.453</td><td>0.678</td><td>0.393</td><td>0.698</td><td>1.541</td><td>0.233</td><td>150 MB</td><td>2.33</td></tr><tr><td>AdaBins (Pruned Randomly 10%)</td><td>0.322</td><td>0.602</td><td>0.807</td><td>0.354</td><td>0.550</td><td>1.334</td><td>0.182</td><td>268 MB</td><td>1.57</td></tr><tr><td>AdaBins (Pruned Randomly 30%)</td><td>0.257</td><td>0.505</td><td>0.712</td><td>0.744</td><td>1.490</td><td>1.575</td><td>0.218</td><td>210 MB</td><td>2.21</td></tr><tr><td>AdaBins (Pruned Randomly 50%)</td><td>0.124</td><td>0.294</td><td>0.494</td><td>1.204</td><td>3.560</td><td>2.420</td><td>0.307</td><td>150 MB</td><td>2.48</td></tr><tr><td>AdaBins (Quantized FP16)</td><td>0.579</td><td>0.890</td><td>0.973</td><td>0.240</td><td>0.214</td><td>0.739</td><td>0.124</td><td>150 MB</td><td>2.7</td></tr><tr><td>AdaBins (Quantized FP8)</td><td>0.138</td><td>0.304</td><td>0.503</td><td>0.463</td><td>0.908</td><td>1.757</td><td>0.300</td><td>77 MB</td><td>4.1</td></tr></tbody></table>

## 2. Related Work

This section reviews recent lightweight monocular depth estimation (MDE) models and their design strategies. Although large-scale networks achieve state-of-the-art accuracy, their heavy computation and memory demands hinder real-time or embedded use. Compact architectures aim to improve efficiency but often trade accuracy for this. Below, we summarize representative lightweight models and their key contributions.
## 2.1. FastDepth [19]

FastDepth, an early real-time MDE model, uses MobileNet as its encoder [20], leveraging depthwise separable convolutions to cut computation compared to standard backbones [21]. The decoder comprises five up-sampling blocks with depthwise convolutions and nearest-neighbor interpolation. Convolution precedes up-sampling to reduce overhead, and skip connections link the encoder and decoder via element-wise addition [22]. These choices yield accuracy comparable to larger models with very low latency, making FastDepth ideal for embedded devices.

## 2.2. JointDepth [23]

JointDepth tackles depth estimation and semantic segmentation together in one lightweight model. The backbone is MobileNet-v2 [20], followed by a Light-Weight RefineNet [24] that uses Chained Residual Pooling (CRP) to gather context. In the final CRP block, the authors replace the conventional convolution blocks with depthwise convolutions, reducing the computational complexity from over 14 to 6.5 GFLOPs for an input size of 640 × 480 [21]. After the decoder, two compact heads produce segmentation and depth outputs. Training uses a combined loss that merges softmax cross-entropy with an inverse Huber term to balance the multitask objectives [25].

## 2.3. TuMDE [26]

TuMDE enhances an encoder–decoder design with attention and refinement modules for high-resolution prediction. Its ResNet-50 encoder uses dilated convolutions [27] and coordinate attention [28] to encode spatial dependencies along horizontal and vertical directions. The decoder performs three progressive up-sampling steps and fuses multi-scale features from encoder stages via up-projection and concatenation. A refinement module with channel reweighting produces smoother, more accurate depth maps, showing the benefits of attention and multi-scale fusion.

## 2.4. YinMDE [29]

YinMDE couples depth prediction with 3D point cloud reconstruction. Depth maps from the encoder–decoder are projected to point clouds using the pinhole camera model [30]. Training combines pixel-wise supervision with a Virtual Normal (VN) loss [29] that enforces long-range geometric consistency, aligning pixel-level precision with global 3D structure.

## 2.5. DaNet [31]

DaNet merges compact convolutions and transformer reasoning [32]. Using an EfficientNet-B0 encoder [33] with MBConv blocks [34], it balances efficiency and representation power. A Pyramid Scene Transformer (PST) captures global context by dividing inputs into multi-scale patches processed by adaptive convolutions and transformer encoders. The decoder employs four residual up-sampling blocks for low latency and for detailed reconstruction. Depth is predicted through discretization with dynamically learned bin centers and integrated probabilities, achieving high accuracy with low computation.

## 2.6. GuideDepth [35]

GuideDepth is designed for efficient, high-fidelity up-sampling by leveraging a DDRNet-23-slim 126 encoder, which enables real-time inference [36]. The decoder stacks three Guided Up-sampling Blocks (GUBs) that fuse decoder features with multi-resolution image guidance via bilinear up-sampling, stacked convolutions, and adaptive channel weighting. The final GUB runs at full resolution to sharpen edges and preserve fine details with little added cost.
## 2.7. LMDepth [37]

LMDepth integrates sequence modeling into MDE. It pairs a MobileNetV2 encoder and Modified Pyramid Spatial Pooling (MPSP) [38] with a decoder composed of Depth Mamba Blocks (DMBs) [39] within the VisionMamba framework [40]. DMBs provide efficient, linear-time long-range reasoning while refining depth estimates; final predictions combine discretized bins with refined probability distributions to preserve structural consistency and efficiency.

While these methods reduce complexity through architectural innovations—ranging from efficient convolutions to hybrid CNN-transformer designs—few optimize the training paradigm itself. Leveraging large models to supervise compact ones remains underexplored. This work follows that direction, using knowledge distillation to train an efficient student network under both ground-truth and teacher supervision, prioritizing training strategy over structural modification.

# 3. Methodology

## 3.1. *Model Overview*

The proposed network is an optimized model seeking a balance between predictive accuracy and computational efficiency by integrating an EfficientNet encoder [33] with a DenseNet-inspired decoder [41]. It extracts high-level semantic information from the input image using a lightweight convolutional backbone and progressively constructs the final depth map through connected blocks (see Figure 3).

The encoder is built on EfficientNet-B1, which keeps the same MBConv blocks with squeeze-and-excitation modules as EfficientNet-B0 but scales the network in depth, width, and input size [34]. It begins with a stem convolution and then stacks MBConv layers with different kernel sizes and expansion ratios, gradually reducing the spatial dimensions while adding richer channel features. Compared to B0, the B1 variant is only slightly larger but provides stronger feature representations thanks to its deeper and wider architecture. To make it suitable for depth estimation, we removed the classification head (fully connected layer, global pooling, and dropout) so that spatial details are preserved instead of being compressed into a single vector. We replaced the original head with a simple $1 \times 1$ convolution followed by batch normalization to produce a 320-channel bottleneck feature map (reduced from 1280). This reduces redundancy while preserving salient information, yielding a lighter encoder better suited for dense prediction. The 320-channel feature map is then passed to the decoder.

The decoder is designed to reconstruct a high-resolution depth map from the compressed feature representations produced by the encoder. It comprises four consecutively stacked *DenseDecoderBlocks*, which serve as the decoder's primary processing units. Each decoder block is connected to the corresponding encoder stage via a residual skip connection; these connections reintroduce high-resolution spatial detail preserved during encoding and thereby improve the structural fidelity of the generated depth map.

The **DenseDecoderBlock** starts by using multi-scale feature extraction, implemented through the use of three depthwise separable convolutions. The operation at each level applies a spatial depthwise convolution using a kernel size of 3 × 3, followed by a 1 × 1 pointwise convolution, thereby combining knowledge from different feature maps. ReLU activations, together with batch normalization, are used to ensure nonlinearity and to stabilize training. The resulting feature maps are then concatenated along the channel axis to form a densely connected representation that encourages feature reuse and improves gradient flow. The concatenated tensor is subsequently up-sampled using bilinear interpolation to restore spatial resolution. To preserve fine-grained details, the up-sampled features are concatenated with the corresponding higher-resolution encoder features via a residual
skip connection. Finally, a concluding convolutional layer with batch normalization refines the fused representation before it is forwarded to the next decoding block. An illustration of the **DenseDecoderBlock** is provided in Figure 4.

<figure><img src="image_5.png" /><figcaption>**Figure 3.** Architecture of Model with learning process: The student network (top) learns to mimic intermediate feature representations of the teacher (bottom) while simultaneously being supervised on the ground-truth depth map using a scale-invariant loss, enabling efficient and accurate depth prediction with a compact architecture.</figcaption></figure>

The adoption of depthwise separable convolutions is a deliberate design choice intended to balance model efficiency and representational capacity. By factorizing a standard convolution into a spatial, per-channel depthwise filtering operation followed by a 1 × 1 pointwise projection, the block substantially reduces parameters while retaining the capacity to model local spatial structure and inter-channel dependencies. The proposed skip-connection topology combined with the presented decoding scheme promotes computational efficiency without materially degrading the expressive power required for accurate depth reconstruction.
<figure><img src="image_6.png" /><figcaption>**Figure 4.** illustration of the proposed DenseDecoderBlock. The design emphasizes dense feature connections, efficient multi-scale representation through separable convolutions, and smooth integration with encoder features to recover fine spatial details during decoding.</figcaption></figure>

## 3.2. Training Process

The training process of the proposed lightweight depth estimation network is guided by a knowledge distillation paradigm, where a compact student model is trained to approximate the embedding produced by a larger teacher network (we used AdaBins by Farooq Bhat et al., 2021 [18]). This strategy allows the student to inherit structural and semantic knowledge from the teacher while maintaining a significantly lower computational footprint.

The AdaBins architecture consists of an encoder–decoder backbone that generates dense feature representations with 128 channels, followed by an adaptive binning module (see Figure 3). The adaptive binning module creates a feature map with 256 channels and 256 bin centers. The feature map represents the probability of each pixel belonging to a given depth range, where the bin centers are used to estimate the length of these ranges. The probability map (feature map in Figure 3) and the bin centers are linearly summed to produce the final depth map as follows:

$$D_{(i,j)} = \sum_{k=0}^{\text{bins}} w_{(i,j)}^{k} c^{k} \qquad (1)$$

Our lightweight student with its encoder–decoder is trained to produce a 256-channel feature map intended to carry the same structural clues as the teacher’s probability map. A single 1 × 1 convolution (the final decoding head), followed by a pointwise nonlinearity, then maps that 256-channel representation to a scalar depth map. Indeed, this 1 × 1 convolution head acts as a linear combinator, where it learns to mix the channel-wise
responses (which are trained to behave like implicit bin probabilities) into a depth value without computing explicit bin centers. To transfer AdaBins' structural priors, we force the student's 256-channel decoder output to mimic the teacher's 256-channel probability distribution at every pixel. Let P denote the teacher's probability map and let z_s be the student's raw decoder output. We convert the student feature map (generated by its decoder) into probabilities using log-softmax:

$$ \log Q_i = z_{s,i} - \log \sum_j e^{z_{s,j}} \qquad (2) $$

We then compute the Kullback-Leibler divergence:

$$ L_{KL} = \frac{1}{n} \sum_{i=1}^{n} P_i(\log P_i - \log Q_i) \qquad (3) $$

Minimizing $L_{KL}$ encourages the student channels to distribute activation mass across channels in a manner that matches the teacher's bin-wise probabilities, thereby transferring semantic and structural cues without bin-center integration.

We compute $\log Q$ via a single log-softmax call for two practical reasons. First, numerical stability: log-softmax uses the log-sum-exp trick and avoids overflow/underflow that can arise when exponentiating large logits or taking logs of tiny probabilities. Second, gradient stability and efficiency: the KL term naturally uses log-probabilities, so producing $\log Q$ directly avoids an extra log on small softmax outputs (which can amplify numerical noise) and yields cleaner gradients during backpropagation. The distillation temperature $\tau$, which normally scales the logits before converting to probability distributions, is set to $\tau = 1$ in our experiments [42].

In addition to distillation, the student is directly supervised on the predicted depth map using a scale-invariant loss function, as employed in AdaBins [3,18]. This formulation addresses the ambiguity of absolute scale in monocular depth estimation by penalizing relative depth inconsistencies rather than absolute errors. Given the predicted depth $\hat{D}$ and ground-truth depth $D$, the scale-invariant loss is defined as

$$ L_{\text{scale}} = \frac{1}{n} \sum_{i} d_{i}^{2} - \frac{1}{n^{2}} \left( \sum_{i} d_{i} \right)^{2} \qquad (4) $$

where $d_i = \log(\hat{D}_i) - \log(D_i)$ and n is the number of valid pixels. This loss is robust to global scale differences and is particularly effective in monocular settings.

The scale-invariant supervision and feature distillation terms are unified in the final training objective:

$$ L_{\text{total}} = \alpha L_{\text{scale}} + \beta L_{\text{KL}} \qquad (5) $$

where $\alpha$ and $\beta$ are weighting parameters that control the relative importance of scale-consistent depth estimation and representation alignment with the teacher. The suggested training approach combines scale-invariant supervision and distillation at the feature level in an efficient manner such that the student network benefits from the ability of AdaBins for structural reasoning without associated computational cost. The balanced approach therefore guarantees scale ambiguity robustness and efficiency, offering an original framework for practical, real-time monocular depth estimation.

## 4. Experiment

This section begins with a concise presentation of the datasets used and the evaluation metrics, followed by a detailed description of the technical implementation, and finally the
obtained experimental results. We then present a quantitative comparison with state-of-the-art methods.

## 4.1. Datasets

The KITTI dataset consists of stereo image pairs and 3D laser scans of outdoor driving scenes, collected using a LiDAR sensor [43] mounted on a moving vehicle [44]. The RGB images have a resolution of approximately 1241 × 376 pixels, while the corresponding depth maps are highly sparse, with measurable distances extending up to 80 m. For training, our model is trained on a subset of approximately 23,000 images and evaluated on 697 images. This split follows the protocol defined by Eigen et al. [3], where uncorrelated frames are selected from different scenes to avoid temporal correlation and encourage the model to predict each pixel's depth independently, as measured by the sensor. During evaluation, we apply a validity mask that includes only pixels with depth values in the range of 10⁻³ to 80 m, and we remove invalid boundary regions using the cropping mask proposed by Garg et al. [45].

The NYU Depth v2 dataset provides a large-scale benchmark of indoor RGB-D images, recorded at a resolution of 640 × 480 pixels using the Microsoft Kinect sensor [46]. It contains approximately 120,000 training samples and 654 test images [47]. Similar to the KITTI dataset, and to ensure fair comparison with previous works, we utilize the train/test split defined by Eigen et al. [3]. The valid depth range is set between 10⁻³ and 10 m, following the central cropping strategy of Eigen et al. [3]. During inference, the final prediction is computed as the average of the model's outputs on the original and horizontally flipped images, an augmentation strategy commonly employed to enhance robustness and consistency in depth estimation.

The preprocessing, which includes exclusively a rotation of 10°, horizontal flipping, and normalization of the RGB images, is based on the Z-score approach, where a unique mean and standard deviation are used for all images [48]. The rotation and flipping are performed randomly, while normalization is applied to all images. It is worth noting that the dataset splits used here follow the Eigen split for both KITTI and NYUv2 datasets, with the cropping strategies proposed by Eigen et al. [3] and Garg et al. [45], which serve as common references for numerous subsequent works [3,7,8,18,45].

## 4.2. Metrics

To assess the performance of our model against previous lightweight MDE methods designed for resource-limited devices, we use the same evaluation metrics adopted in prior works. Specifically, we employ the standard metrics commonly used in depth estimation studies, defined as follows: Average relative error (REL):

$$ REL = \frac{1}{n} \sum_{i=1}^{n} \frac{|y_i - \hat{y}_i|}{y_i} \qquad (6) $$

Root mean squared error (RMS):

$$ RMS = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2} \qquad (7) $$

Average logarithmic error (log₁₀):

$$ \log_{10} = \frac{1}{n} \sum_{i=1}^{n} |\log_{10}(y_i) - \log_{10}(\hat{y}_i)| \qquad (8) $$
Threshold accuracy ($\delta_m$):

$$ \delta_m = \max\left(\frac{y_i}{\hat{y}_i}, \frac{\hat{y}_i}{y_i}\right) < 1.25^m \quad (9) $$

For the KITTI dataset we use the squared relative difference (Sq. Rel) instead of the log₁₀ metrics:

$$ \text{Sq. Rel} = \frac{1}{n} \sum_{i=1}^{n} \frac{(y_i - \hat{y}_i)^2}{y_i} \quad (10) $$

Here, $y_i$ is a pixel in the depth map $y$, $\hat{y}_i$ is the corresponding pixel in the predicted depth map $\hat{y}$, and $n$ is the total number of pixels with valid values (in the range of the prediction), while invalid pixels are not considered in the evaluation process.

The RMS measures the magnitude of error in absolute terms. The relative error measures that evaluate proportional deviations across scales include REL and Sq. Rel, while Log₁₀ captures logarithmic differences, providing a scale-invariant perspective. The accuracy metric $\delta_m$ quantifies the proportion of predictions within increasing tolerance bounds, providing an interpretable measure of reliability. Together, these metrics provide a balanced evaluation that accounts for both overall error and prediction accuracy.

In addition to these metrics, we employ an uncertainty measurement to assess the confidence range of our models using the 95% confidence interval (CI₉₅), computed as

$$ CI_{95} = 1.96 \times \frac{\sigma}{\sqrt{n}}, $$

where $\sigma$ denotes the standard deviation of the per-image metric values, and $n$ represents the number of samples in the evaluation set. This formulation estimates the range within which the true mean performance lies with 95% probability, thereby reflecting the reliability and stability of the reported results. It is important to note that $CI_{95}$ was not employed in previous works as part of their evaluation metrics; therefore, we report it only for the models that we were able to evaluate locally.

In addition to accuracy-based error metrics, we report efficiency measures that characterize the computational cost and practicality of the model:

* **FLOPs (Floating-Point Operations):** The number of arithmetic operations required per forward pass, reflecting overall computational complexity;

* **Number of Parameters:** The total number of trainable weights, which indicates model size and memory footprint;

* **Frames Per Second (FPS):** The throughput of the model, expressed as the number of depth maps processed per second at an input resolution of (256 × 256) measured exclusively on low-computing-power devices.

### 4.3. Implementation Details

The distilled model is implemented using PyTorch [49], which provides a flexible infrastructure facilitating efficient tensor computations and GPU acceleration. The optimization procedure is carried out using the AdamW optimizer [50] with a maximum learning rate of 3.75 × 10⁻⁴ and a weight decay of 0.01, designed to prevent overfitting through explicit regularization of the model parameters [51]. The training methodology follows the One Cycle Learning Rate policy [52], where the learning rate starts from 10⁻⁶, then incrementally increases to its maximum value 3.75 × 10⁻⁴ and linearly decreases to its starting value, encouraging faster convergence and improved generalization over 100 training epochs.
The loss is regularized with $\alpha = 10^2$ and $\beta = 10^{-5}$ (Equation (5)), which balances the contributions of each of the error components during optimization. The model is trained with a single NVIDIA RTX 3060 GPU (NVIDIA Corporation, Santa Clara, CA, USA) with 12 GB of memory and a mini-batch size of 6. To deploy in a resource-limited environment, the learned model was tested on a Raspberry Pi 4 board (Raspberry Pi Foundation, Cambridge, UK) [53] with 2 GB of memory, powered by an ARM Cortex-A72 1.5 GHz (ARM Holdings, Cambridge, UK) quad-core CPU, running Ubuntu Server with Python 3.13 and PyTorch 2.8. Latency evaluation uses only the onboard CPU without GPU acceleration, allowing us to assess the real-world usability of the model under low-computing-power conditions.

## 4.4. Results

Tables 2 and 3 present the quantitative results on the NYUv2 and KITTI datasets, respectively, while Figures 5 and 6 illustrate qualitative predictions. On both benchmarks, our model shows clear improvements across nearly all accuracy and error metrics, surpassing existing lightweight methods. In particular, it achieves higher accuracies and lower errors on NYUv2, and it demonstrates strong generalization to outdoor driving scenes on KITTI.

<table><caption>**Table 2.** Comparison with other representative lightweight depth estimation methods on the NYUDv2 dataset under **high-resolution** input (480 × 640). The best results are highlighted in bold, and the second-best results are underlined. FLOPs are computed for a single forward pass and the FPS is measured exclusively at 256 × 256 resolution with the Raspberry Pi device.</caption><thead><tr><th>Method</th><th>Params (M) ↓</th><th>FLOPs (G) ↓</th><th>FPS ↑</th><th>δ<sub>1</sub> ↑</th><th>δ<sub>2</sub> ↑</th><th>δ<sub>3</sub> ↑</th><th>REL ↓</th><th>log<sub>10</sub> ↓</th><th>RMS ↓</th></tr></thead><tbody><tr><td>FastDepth [19]</td><td>3.9</td><td>4.62</td><td>1.12</td><td>0.778</td><td>0.942</td><td>0.987</td><td>0.142</td><td>-</td><td>0.591</td></tr><tr><td>JointDepth [23]</td><td>3.0</td><td>6.49</td><td>0.58</td><td>0.790</td><td>0.955</td><td>0.990</td><td>0.149</td><td>-</td><td>0.565</td></tr><tr><td>TuMDE [26]</td><td>4.6</td><td>4.60</td><td>-</td><td>0.772</td><td>0.948</td><td>0.987</td><td>0.162</td><td>0.067</td><td>0.536</td></tr><tr><td>YinMDE [29]</td><td>2.7</td><td>15.80</td><td>-</td><td>0.813</td><td>0.958</td><td>0.991</td><td>0.135</td><td>0.060</td><td>-</td></tr><tr><td>DaNet [31]</td><td>8.2</td><td>5.51</td><td>0.33</td><td>0.829</td><td>0.963</td><td>0.991</td><td>0.140</td><td>0.059</td><td>0.487</td></tr><tr><td>GuideDepth [35]</td><td>5.8</td><td>10.47</td><td>-</td><td>0.844</td><td>0.971</td><td>0.992</td><td>0.129</td><td>0.055</td><td>0.449</td></tr><tr><td>LMDepth [37]</td><td>2.9</td><td>2.77</td><td>-</td><td>0.854</td><td>0.977</td><td><strong>0.995</strong></td><td>0.123</td><td>0.052</td><td>0.438</td></tr><tr><td>Ours (B1)</td><td>6.82</td><td>22.53</td><td>0.38</td><td><u>0.866</u></td><td><strong>0.979</strong></td><td><strong>0.995</strong></td><td><u>0.119</u></td><td><u>0.050</u></td><td><strong>0.419</strong></td></tr><tr><td>Ours (FastDepth)</td><td>4.9</td><td>4.84</td><td>1.02</td><td><strong>0.873</strong></td><td><strong>0.979</strong></td><td><strong>0.995</strong></td><td><strong>0.118</strong></td><td><strong>0.049</strong></td><td><strong>0.429</strong></td></tr></tbody></table>

<table><caption>**Table 3.** Comparison with other representative lightweight depth estimation methods on the KITTI dataset under **high-resolution** input (384 × 1260). The best results are highlighted in bold, and the second-best results are underlined. FLOPs are computed for a single forward pass and the FPS is measured exclusively at 256 × 256 resolution with the Raspberry Pi device.</caption><thead><tr><th>Method</th><th>Params (M) ↓</th><th>FLOPs (G) ↓</th><th>FPS ↑</th><th>δ<sub>1</sub> ↑</th><th>δ<sub>2</sub> ↑</th><th>δ<sub>3</sub> ↑</th><th>REL ↓</th><th>Sq. Rel ↓</th><th>RMS ↓</th></tr></thead><tbody><tr><td>FastDepth [19]</td><td>3.9</td><td>6.17</td><td>1.12</td><td>0.825</td><td>0.961</td><td>0.991</td><td>0.136</td><td>0.714</td><td>4.33</td></tr><tr><td>TuMDE [26]</td><td>4.6</td><td>4.5</td><td>-</td><td>0.770</td><td>0.910</td><td>0.980</td><td>0.145</td><td>0.630</td><td>6.50</td></tr><tr><td>DaNet [31]</td><td>8.2</td><td>7.85</td><td>0.33</td><td>0.920</td><td>0.981</td><td>0.995</td><td>0.086</td><td>0.411</td><td>3.52</td></tr><tr><td>GuideDepth [35]</td><td>5.8</td><td>16.75</td><td>-</td><td>0.882</td><td>0.972</td><td>0.991</td><td>0.109</td><td>-</td><td>4.10</td></tr><tr><td>LMDepth [37]</td><td>2.9</td><td>4.05</td><td>-</td><td><u>0.923</u></td><td><u>0.988</u></td><td><u>0.997</u></td><td><u>0.079</u></td><td><u>0.352</u></td><td><u>3.28</u></td></tr><tr><td>Ours (B1)</td><td>6.82</td><td>35.16</td><td>0.38</td><td><strong>0.949</strong></td><td><strong>0.992</strong></td><td><strong>0.998</strong></td><td><strong>0.070</strong></td><td><strong>0.285</strong></td><td><strong>2.860</strong></td></tr><tr><td>Ours (FastDepth)</td><td>4.9</td><td>6.76</td><td>1.02</td><td>0.924</td><td>0.974</td><td><strong>0.998</strong></td><td><u>0.079</u></td><td>0.454</td><td>3.543</td></tr></tbody></table>

Conversely, latency and inference speed remain less competitive compared to ultra-lightweight baselines due to the architectural design. To mitigate this limitation, we adopt FastDepth as a student model and apply our proposed distillation strategy, which preserves most of the accuracy gains while significantly improving efficiency, yielding a favourable trade-off for real-world deployment. The results are shown in Tables 2 and 3.

Moreover, to evaluate the proposed distilled model, we also compare its performance against pruned and quantized versions of the teacher model. As reported in Table 4, our distilled model achieves competitive results with the full-size AdaBins model, while
delivering higher quantitative accuracy, smaller model size, and faster inference than the
pruned and quantized models on the NYUv2 dataset.

<figure><img src="image_7.png" /><figcaption>**Figure 5.** Qualitative comparison of different models trained using our distillation approach on the NYUDv2 dataset [23].</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Figure 6.** Qualitative comparison of different models trained using our distillation approach on the KITTI dataset [18,23].</figcaption></figure>

We tested the generalization performance of the trained models (based on B1 and FastDepth) by training each model on one dataset and evaluating it on an unseen dataset. In our case, we evaluated models trained on the NYUv2 dataset using the SUNRGB-D dataset, which is an indoor dataset containing 5050 image-depth pairs with highly dense depth maps [54]. The depth range for this evaluation is from 0.001 to 10, and any values
outside this range are excluded from the evaluation. The preprocessing step includes only input normalization using Z-score. Table 5 illustrates the results compared to those with the AdaBins model used as the teacher.

<table><caption>**Table 4.** Performance comparison between AdaLite, the AdaBins model, and its compressed models on the NYUv2 dataset (same configurations as in Table 1). Arrows show metric direction: in the top row, ↑ means higher is better; in the bottom row, ↓ means lower is better.</caption><thead><tr><th>Model</th><th>δ<sub>1</sub> ↑</th><th>Rel ↓</th><th>RMS ↓</th><th>log10 ↓</th><th>Model Size ↓</th><th>FPS ↑</th></tr></thead><tbody><tr><td>AdaBins (Baseline)</td><td>0.902 ± 0.010</td><td>0.103 ± 0.005</td><td>0.369 ± 0.019</td><td>0.044 ± 0.002</td><td>299 MB</td><td>1.2</td></tr><tr><td>AdaBins (Pruned Magnitude 10%)</td><td>0.857 ± 0.012</td><td>0.118 ± 0.005</td><td>0.428 ± 0.021</td><td>0.056 ± 0.009</td><td>268 MB</td><td>1.5</td></tr><tr><td>AdaBins (Pruned Magnitude 20%)</td><td>0.838 ± 0.013</td><td>0.122 ± 0.006</td><td>0.453 ± 0.022</td><td>0.055 ± 0.002</td><td>239 MB</td><td>2.0</td></tr><tr><td>AdaBins (Pruned Magnitude 40%)</td><td>0.234 ± 0.021</td><td>0.352 ± 0.008</td><td>1.360 ± 0.058</td><td>0.204 ± 0.006</td><td>179 MB</td><td>2.2</td></tr><tr><td>AdaBins (Quantized FP16)</td><td>0.579 ± 0.016</td><td>0.240 ± 0.010</td><td>0.739 ± 0.031</td><td>0.125 ± 0.023</td><td>150 MB</td><td>2.7</td></tr><tr><td>AdaBins (Quantized FP8)</td><td>0.138 ± 0.027</td><td>0.463 ± 0.027</td><td>1.757 ± 0.059</td><td>0.301 ± 0.102</td><td>77 MB</td><td><u>4.1</u></td></tr><tr><td>Ours (B1)</td><td><u>0.866 ± 0.011</u></td><td><u>0.119 ± 0.005</u></td><td><u>0.419 ± 0.018</u></td><td><u>0.050 ± 0.003</u></td><td>29 MB</td><td>2.10</td></tr><tr><td>Ours (FastDepth)</td><td><b>0.873 ± 0.010</b></td><td><b>0.118 ± 0.005</b></td><td><b>0.429 ± 0.020</b></td><td><b>0.049 ± 0.003</b></td><td>19 MB</td><td><b>13.54</b></td></tr></tbody></table>

<table><caption>**Table 5.** Cross-dataset generalization results on the SUNRGBD dataset with 95% confidence intervals (CI<sub>95s</sub>). Our models are trained on NYUv2 and tested under the same environment with the others.</caption><thead><tr><th>Model</th><th>δ<sub>1</sub></th><th>δ<sub>2</sub></th><th>δ<sub>3</sub></th><th>Rel</th><th>Sq. Rel</th><th>RMS</th><th>log<sub>10</sub></th></tr></thead><tbody><tr><td>AdaBins</td><td>0.765 ± 0.006</td><td>0.942 ± 0.004</td><td>0.982 ± 0.002</td><td>0.160 ± 0.003</td><td>0.112 ± 0.005</td><td>0.481 ± 0.009</td><td>0.070 ± 0.001</td></tr><tr><td>AdaBins + Pruning (Magnitude 20%)</td><td><u>0.627 ± 0.007</u></td><td><u>0.887 ± 0.005</u></td><td><u>0.968 ± 0.003</u></td><td><u>0.196 ± 0.003</u></td><td><u>0.166 ± 0.005</u></td><td><u>0.643 ± 0.011</u></td><td><u>0.093 ± 0.001</u></td></tr><tr><td>AdaBins + Pruning (Gradient 20%)</td><td>0.419 ± 0.006</td><td>0.706 ± 0.006</td><td>0.869 ± 0.004</td><td>0.368 ± 0.006</td><td>0.437 ± 0.010</td><td>0.989 ± 0.015</td><td>0.149 ± 0.002</td></tr><tr><td>Ours (B1)</td><td><b>0.738 ± 0.007</b></td><td><b>0.932 ± 0.004</b></td><td><b>0.981 ± 0.002</b></td><td><b>0.176 ± 0.004</b></td><td><b>0.128 ± 0.005</b></td><td><b>0.512 ± 0.009</b></td><td><b>0.074 ± 0.001</b></td></tr><tr><td>Ours (FastDepth)</td><td>0.622 ± 0.006</td><td>0.869 ± 0.005</td><td>0.957 ± 0.003</td><td>0.237 ± 0.005</td><td>0.213 ± 0.007</td><td>0.654 ± 0.011</td><td>0.097 ± 0.002</td></tr></tbody></table>

## 4.5. Ablation Study

To better understand how our methodology behaves across different configurations, we conducted an ablation study by systematically varying the distillation parameter ($\beta$ in Equation (5)). Instead of presenting accuracy and cosine similarity in separate plots, we report both measures in a single figure (Figure 7) to highlight their joint behaviour. Specifically, we vary $\beta$ from 1 down to $10^{-7}$ and analyze the resulting trade-off between prediction accuracy and the cosine similarity of student–teacher feature representations. It can be observed that predictive accuracy is relatively poor at $\beta = 1$, which we attribute to the student model’s tendency to copy intermediate teacher outputs rather than refining depth predictions. As $\beta$ decreases, accuracy steadily improves and reaches its peak near $\beta = 10^{-6}$. In contrast, cosine similarity decreases with lower $\beta$, indicating that while the student departs from directly mimicking the teacher’s probability distribution, it achieves higher-quality depth predictions. This combined analysis reveals the balance between effective feature distillation and accurate depth estimation.

We extend this exploration further by using the model designed on the B1 architecture without relying on anything other than the scale-invariant loss Equation (4) and without implementing intermediate alignment’s integration in AdaBins as a guiding element. The results corresponding to this configuration, along with the baseline FastDepth [19] model and its distilled version, can be seen in Table 6. This assessment highlights the role that our distillation scheme plays in enhancing depth estimation without losing out on model efficiency.
<table><caption>**Table 6.** Ablation study illustrating the effect of distillation on model performance. Models are trained and evaluated on the NYUv2 dataset at full resolution. To remove the effect of distillation, we set $\beta = 0$.</caption><thead><tr><th>Method</th><th>Rel ↓</th><th>RMS ↓</th><th>log<sub>10</sub> ↓</th><th>δ<sub>1</sub> ↑</th><th>δ<sub>2</sub> ↑</th></tr></thead><tbody><tr><td>B1 [33]<br>+ Distillation</td><td>0.124<br><strong>0.119</strong></td><td>0.446<br><strong>0.419</strong></td><td>0.057<br><strong>0.050</strong></td><td>0.841<br><strong>0.866</strong></td><td>0.977<br><strong>0.979</strong></td></tr><tr><td>FastDepth [19]<br>+ Distillation</td><td>0.142<br><strong>0.118</strong></td><td>0.591<br><strong>0.429</strong></td><td>–</td><td>0.778<br><strong>0.873</strong></td><td>0.942<br><strong>0.979</strong></td></tr></tbody></table>

<figure><img src="image_9.png" /><figcaption>**Figure 7.** Accuracy and cosine similarity across different values of KL term $\beta$. Higher loss terms ($\beta$) preserve stronger student–teacher alignment but limit accuracy, while lower values improve depth prediction performance at the cost of reduced feature similarity. An optimal balance between the cosine similarity and the prediction accuracy is observed around $10^{-3}$ to $10^{-6}$.</figcaption></figure>

## 5. Conclusions

In this work, we introduced AdaLite, a lightweight knowledge distillation approach for monocular depth estimation aimed at real-time use on resource-limited devices. The method is designed to keep a balance between accuracy and efficiency by using a dual-supervision strategy, where the student model learns from both teacher features and ground-truth depth. We showed that small networks can still capture the strengths of large models without the need for pruning or quantization. Our experiments on NYUv2 and KITTI demonstrate that AdaLite outperforms current lightweight baselines, while keeping inference fast and model size small. The EfficientNet-B1 and FastDepth students compress the original teacher models by up to 90%, yet retain comparable state-of-the-art accuracy. This underlines the value of distillation as a way to close the gap between high-capacity models and practical deployment on limited hardware. In later stages, we see potential to extend our system by incorporating quantization-aware training, deploying the model in real embedded applications, and adopting more powerful teacher models, such as IEBins [9] or metric-scale depth networks [55], in the hope of constructing more compact and notably more capable student models. Continuing down this path will also enable the improvement of reliable and expedient depth estimation and make it more accessible for use in real-world applications.
**Author Contributions:** M.C.Z.: Conceptualization, Methodology, Writing—original draft preparation, Writing—review and editing; M.E.: Conceptualization, Visualization, Supervision; N.T. and K.K.: Reviewing, Supervision; S.A.E.M.A.: Supervision. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** The data used in this study is publicly available: KITTI http://www.cvlabs.net/datasets/kitti/ and NYUv2 https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html. The code of AdaLite is available at: https://github.com/Chaouki-AI/AdaLite (accessed on 10 November 2025).

**Acknowledgments:** This work was supported by the Algerian Directorate General for Scientific Research and Technological Development (DGRSDT).

**Conflicts of Interest:** The authors declare no conflicts of interest.

## References

1. Chen, P.Y.; Liu, A.H.; Liu, Y.C.; Wang, Y.C.F. Towards Scene Understanding: Unsupervised Monocular Depth Estimation With Semantic-Aware Representation. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15-20 June 2019; pp. 2619-2627. [CrossRef]

2. Diamantas, S.C.; Oikonomidis, A.; Crowder, R.M. Depth estimation for autonomous robot navigation: A comparative approach. In Proceedings of the 2010 IEEE International Conference on Imaging Systems and Techniques, Thessaloniki, Greece, 1-2 July 2010; pp. 426-430. [CrossRef]

3. Eigen, D.; Puhrsch, C.; Fergus, R. Depth map prediction from a single image using a multi-scale deep network. In Proceedings of the 28th International Conference on Neural Information Processing Systems, NIPS'14, Montreal, QC, Canada, 8-13 December 2014; Volume 2, pp. 2366-2374.

4. Fu, H.; Gong, M.; Wang, C.; Batmanghelich, K.; Tao, D. Deep Ordinal Regression Network for Monocular Depth Estimation. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18-22 June 2018; pp. 2002-2011.

5. Cao, Y.; Wu, Z.; Shen, C. Estimating Depth From Monocular Images as Classification Using Deep Fully Convolutional Residual Networks. IEEE Trans. Circuits Syst. Video Technol. 2018, 28, 3174-3182. [CrossRef]

6. Agarwal, A.; Arora, C. Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention. In Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 2-7 January 2023; pp. 5850-5859. [CrossRef]

7. Bhat, S.F.; Alhashim, I.; Wonka, P. LocalBins: Improving Depth Estimation by Learning Local Distributions. In Proceedings of the Computer Vision-ECCV 2022, Tel Aviv, Israel, 23-27 October 2022; pp. 480-496.

8. Li, Z.; Wang, X.; Liu, X.; Jiang, J. BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation. IEEE Trans. Image Process. 2024, 33, 3964-3976. [CrossRef] [PubMed]

9. Shao, S.; Pei, Z.; Wu, X.; Liu, Z.; Chen, W.; Li, Z. IEBins: Iterative elastic bins for monocular depth estimation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, New Orleans, LA, USA, 10-16 December 2023.

10. Zhu, R.; Song, Z.; Liu, L.; He, J.; Zhang, T.; Zhang, Y. HA-Bins: Hierarchical Adaptive Bins for Robust Monocular Depth Estimation Across Multiple Datasets. IEEE Trans. Circuits Syst. Video Technol. 2024, 34, 4354-4366. [CrossRef]

11. Paleyes, A.; Urma, R.G.; Lawrence, N.D. Challenges in Deploying Machine Learning: A Survey of Case Studies. ACM Comput. Surv. 2022, 55, 1-29. [CrossRef]

12. Cheng, H.; Zhang, M.; Shi, J.Q. A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations. arXiv 2024, arXiv:2308.06767. [CrossRef] [PubMed]

13. Yang, J.; Li, Z.; Feng, Z.; Xie, Y. A Survey On Neural Network Quantization. In Proceedings of the 2025 6th International Conference on Computer Information and Big Data Applications, CIBDA'25, Wuhan, China, 14-16 March 2025; pp. 384-394. [CrossRef]

14. Lopes, A.; Pereira dos Santos, F.; de Oliveira, D.; Schiezaro, M.; Pedrini, H. Computer Vision Model Compression Techniques for Embedded Systems: A Survey. Comput. Graph. 2024, 123, 104015. [CrossRef]

15. Gou, J.; Yu, B.; Maybank, S.J.; Tao, D. Knowledge Distillation: A Survey. Int. J. Comput. Vis. 2021, 129, 1789-1819. [CrossRef]
16. Calderon, N.; Mukherjee, S.; Reichart, R.; Kantor, A. A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training. arXiv 2023, arXiv:2305.02031. [CrossRef]

17. Qin, D.; Bu, J.J.; Liu, Z.; Shen, X.; Zhou, S.; Gu, J.J.; Wang, Z.H.; Wu, L.; Dai, H.F. Efficient Medical Image Segmentation Based on Knowledge Distillation. IEEE Trans. Med. Imaging 2021, 40, 3820-3831. [CrossRef]

18. Bhat, S.F.; Alhashim, I.; Wonka, P. AdaBins: Depth Estimation Using Adaptive Bins. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20-25 June 2021; pp. 4009-4018.

19. Wofk, D.; Ma, F.; Yang, T.J.; Karaman, S.; Sze, V. FastDepth: Fast Monocular Depth Estimation on Embedded Systems. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20-24 May 2019; pp. 6101-6108. [CrossRef]

20. Dong, K.; Zhou, C.; Ruan, Y.; Li, Y. MobileNetV2 Model for Image Classification. In Proceedings of the 2020 2nd International Conference on Information Technology and Computer Application (ITCA), Guangzhou, China, 18-20 December 2020; pp. 476-480. [CrossRef]

21. Chollet, F. Xception: Deep Learning With Depthwise Separable Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21-26 July 2017.

22. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27-30 June 2016; pp. 770-778. [CrossRef]

23. Nekrasov, V.; Dharmasiri, T.; Spek, A.; Drummond, T.; Shen, C.; Reid, I. Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotations. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20-24 May 2019; pp. 7101-7107. [CrossRef]

24. Lin, G.; Milan, A.; Shen, C.; Reid, I. RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21-26 July 2017; pp. 5168-5177. [CrossRef]

25. Han, J.; Jiang, Z.; Feng, G. Monocular depth estimation based on Chained Residual Pooling and Gradient Weighted Loss. In Proceedings of the 2023 3rd International Conference on Consumer Electronics and Computer Engineering (ICCECE), Guangzhou, China, 6-8 January 2023; pp. 278-282. [CrossRef]

26. Xu, H.; Li, F. An efficient monocular depth prediction network using coordinate attention and feature fusion. J. Inf. Process. Syst. 2022, 18, 794-802.

27. Kudo, Y.; Aoki, Y. Dilated convolutions for image classification and object localization. In Proceedings of the 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), Nagoya, Japan, 8-12 May 2017; pp. 452-455. [CrossRef]

28. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, Long Beach, CA, USA, 4-9 December 2017, pp. 6000-6010.

29. Yin, W.; Liu, Y.; Shen, C.; Yan, Y. Enforcing Geometric Constraints of Virtual Normal for Depth Prediction. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October-2 November 2019; pp. 5683-5692. [CrossRef]

30. Sturm, P. Pinhole Camera Model. In Computer Vision: A Reference Guide; Ikeuchi, K., Ed.; Springer International Publishing: Cham, Switzerland, 2021; pp. 983-986. [CrossRef]

31. Sheng, F.; Xue, F.; Chang, Y.; Liang, W.; Ming, A. Monocular Depth Distribution Alignment with Low Computation. In Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA, 23-27 May 2022; pp. 6548-6555. [CrossRef]

32. Kolesnikov, A.; Dosovitskiy, A.; Weissenborn, D.; Heigold, G.; Uszkoreit, J.; Beyer, L.; Minderer, M.; Dehghani, M.; Houlsby, N.; Gelly, S.; et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv 2021, arXiv:2010.11929. [CrossRef]

33. Tan, M.; Le, Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning, PMLR, Long Beach, CA, USA, 9-15 June 2019; Volume 97, pp. 6105-6114.

34. Sandler, M.; Howard, A.; Zhu, M.; Zhmoginov, A.; Chen, L.C. MobileNetV2: Inverted Residuals and Linear Bottlenecks. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18-23 June 2018; pp. 4510-4520. [CrossRef]

35. Rudolph, M.; Dawoud, Y.; Güldenring, R.; Nalpantidis, L.; Belagiannis, V. Lightweight Monocular Depth Estimation through Guided Decoding. In Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA, 23-27 May 2022; pp. 2344-2350. [CrossRef]

36. Ye, B.; Xue, R. Dual Attention Dual-Resolution Networks for Real-Time Semantic Segmentation of Street Scenes. IEEE Access 2025, 13, 588-595. [CrossRef]

37. Long, J.; Zhou, X. LMDepth: Lightweight Mamba-based Monocular Depth Estimation for Real-World Deployment. arXiv 2025, arXiv:2505.00980. [CrossRef]
38. He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. *IEEE Trans. Pattern Anal. Mach. Intell.* **2015**, *37*, 1904–1916. [CrossRef]

39. Gu, A.; Dao, T. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. *arXiv* **2024**, arXiv:2312.00752. [CrossRef]

40. Zhu, L.; Liao, B.; Zhang, Q.; Wang, X.; Liu, W.; Wang, X. Vision mamba: Efficient visual representation learning with bidirectional state space model. In Proceedings of the 41st International Conference on Machine Learning, ICML’24, Vienna, Austria, 21–27 July 2024.

41. Huang, G.; Liu, Z.; van der Maaten, L.; Weinberger, K.Q. Densely Connected Convolutional Networks. *arXiv* **2018**, arXiv:1608.06993. [CrossRef] [PubMed]

42. Hinton, G.; Vinyals, O.; Dean, J. Distilling the Knowledge in a Neural Network. *arXiv* **2015**, arXiv:1503.02531. [CrossRef]

43. Velodyne’s HDL-64E: A High Definition LiDAR Sensor for 3-D Applications. *Online* **2007**. Available online: http://velodynelidar.com/docs/papers/HDL (accessed on 22 October 2024).

44. Geiger, A.; Lenz, P.; Stiller, C.; Urtasun, R. Vision meets robotics: The KITTI dataset. *Int. J. Rob. Res.* **2013**, *32*, 1231–1237. [CrossRef]

45. Garg, R.; Bg, V.K.; Carneiro, G.; Reid, I. Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue. In Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands, 11–14 October 2016; pp. 740–756.

46. Zhang, Z. Microsoft Kinect Sensor and Its Effect. *IEEE MultiMedia* **2012**, *19*, 4–10. [CrossRef]

47. Silberman, N.; Hoiem, D.; Kohli, P.; Fergus, R. Indoor Segmentation and Support Inference from RGBD Images. In Proceedings of the Computer Vision—ECCV 2012, Florence, Italy, 7–13 October 2012; pp. 746–760.

48. Singh, N.; Singh, P. Exploring the effect of normalization on medical data classification. In Proceedings of the 2021 International Conference on Artificial Intelligence and Machine Vision (AIMV), Gandhinagar, India, 24–26 September 2021; pp. 1–5. [CrossRef]

49. Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 8–14 December 2019.

50. Loshchilov, I.; Hutter, F. Decoupled Weight Decay Regularization. In Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA, 6–9 May 2019.

51. Bos, S.; Chug, E. Using weight decay to optimize the generalization ability of a perceptron. In Proceedings of the International Conference on Neural Networks (ICNN’96), Washington, DC, USA, 3–6 June 1996; Volume 1, pp. 241–246. [CrossRef]

52. Smith, L.N.; Topin, N. Super-convergence: Very fast training of neural networks using large learning rates. In Proceedings of the Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, Baltimore, MD, USA, 14–18 April 2019; Volume 11006, p. 1100612. [CrossRef]

53. Prasetyo Adi, P.D.; Mappadang, A.; Armi, N.; Santiko, A.B.; Adiprabowo, T.; Suprapto; Zulkarnain, R.; Wirawan, A. Optimization and Development of Raspberry Pi 4 Model B for the Internet of Things. In Proceedings of the 2023 IEEE 9th Information Technology International Seminar (ITIS), Batu Malang, Indonesia, 18–20 October 2023; pp. 1–6. [CrossRef]

54. Song, S.; Lichtenberg, S.P.; Xiao, J. SUN RGB-D: A RGB-D scene understanding benchmark suite. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 567–576. [CrossRef]

55. Piccinelli, L.; Yang, Y.H.; Sakaridis, C.; Segu, M.; Li, S.; Van Gool, L.; Yu, F. UniDepth: Universal Monocular Metric Depth Estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 16–22 June 2024; pp. 10106–10116.

**Disclaimer/Publisher’s Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.