## 4 Experimental Results
### Proposed Network Implementation on Low-Cost MR Glasses  
The profile and components of Jorjin J/EF Plus AR glasses are described by Figure 9, which contains two major parts: an MR goggle with several sensors and a computation device connected through a USB type C connector. There are several useful sensors built on this MR goggle, including a time-of-flight (ToF) sensor, an RGB camera, a gyroscope, an accelerator and a magnetometer. The resolution of the ToF sensor is 8 \(\times\) 8 and its sensing distance range is roughly within 3 meters, which is comparably low. Our goal is to utilize the LRLQ depth map and RGB images from Jorjins MR glasses to obtain a precise and high-resolution depth map by using the EPSON BO-IC400 platform [35], which adopts a Qualcomm SXR1130 processor with 4 GB memory and 64 GB storage. For the Jorjin AR glasses sensor, the resolutions of captured raw RGB images and depth maps are 640 \(\times\) 480 and 8 \(\times\) 8, respectively. Since the spatial positions of RGB and depth images must be fully aligned, we first resize the raw RGB image into 360 \(\times\) 270 and crop the region \(RGB_{\text{crop}}=\{(x,y)\): 68 \(\leq x\leq\) 191, 23 \(\leq y\leq\) 246\(\}\) to make the cropped color images be 224 \(\times\) 224 RGB images for further alignment with depth maps. We tried the spatial transformer network to align the RGB and depth images. However, this deep learning network not only consumes additional costs but also causes some unpredictable inaccuracies in the alignment. Hence, we adopt a semi-automatic alignment by the aid of easy manual refinement. An alignment result for Jorjin MR glasses data is displayed by Figure 10. Because the calculation of depth-estimation loss directly exploits the high-resolution GT depth maps provided by the NYU Depth V2 dataset and EgoGesture dataset, those high-resolution GT depth maps are downsampled as sensor-labeled 8\(\times\)8 depth maps for training the models of our network. For verifying the feasibility of our model for resource-limited MR glasses, we implement the simplest model with the sole-SimpleUp decoder w/o AdaDRBins. Unity with version 2020.3.29f1 is used as our engine to build the virtual scene on MR glasses. Jorjin provided the software development kit (SDK) for MR glasses on Unity, and we can access the MR sensor data with Unity scripting API in C# language. We use Barracuda Unity 3.0.0 as our model inference library. Barracuda Unity provides a lightweight and cross-platform neural network inference API for Unity. It can inference neural networks in ONNX (Open Neural Network Exchange) format directly on both the CPU and GPU of supported platforms. For indoor scenes, the model is trained on the NYU Depth V2 dataset, and we apply max depth filtering of 3 meters on the depth training samples. For handling the gesture scenes, the model is trained on the EgoGesture dataset and we apply max depth filtering of 1.5 meters on the depth training samples. Figure 11 displays the estimated high-precision depths for indoor and dynamic hand scenes on Jorjin MR glasses by the simplest model with only a SimpleUp decoder without AdaDRBins. There are three types of contents, 8\(\times\)8  
Figure 9: Outlook of Jorjin J/EF Plus AR glasses using USB Type C cable for the connection to ESPON BO-IC 400 glasses-specific controller. depths captured by the ToF sensor, the depth estimation result predicted by our proposed network and the RGB image captured by the Jorjin MR glasses. For the result of indoor scenes, we can see the network roughly understand the depth information of the scene. However, the depth sensing range of the MR glasses is very low. Hence, the image with far objects in the first row is unable to have a good predicted depth map. For hand gesture scenes, since the distance range of hand gestures can fit the short depth sensing range of the ToF sensor, the position and distance of a hand with dynamic gestures can be clearly identified. We believe that with the high-resolution depth information on MR glasses, many applications can be more accurate.