# Holographic Near-eye Display with Real-time Embedded Rendering

Antonin Gilles, Pierre Le Gargasson, Grégory Hocquet, Patrick Gioia

► To cite this version:

Antonin Gilles, Pierre Le Gargasson, Grégory Hocquet, Patrick Gioia. Holographic Near-eye Display with Real-time Embedded Rendering. SA '23: SIGGRAPH Asia 2023, Dec 2023, Sydney, Australia. 10.1145/3610548.3618179. hal-04351328

HAL Id: hal-04351328

https://hal.science/hal-04351328v1

Submitted on 18 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
Holographic near-eye display with real-time embedded rendering

ANTONIN GILLES and PIERRE LE GARGASSON, Institute of Research & Technology b<>com, France

GRÉGORY HOCQUET and PATRICK GIOIA, Orange Labs, France

<figure><img src="image_2.png" /><figcaption>Fig. 1. (A) Proposed holographic near-eye display prototype. (B, C) Optical reconstructions of holograms captured through the prototype with a camera focusing on the foreground, illuminated using (B) laser diodes and (C) superluminescent diodes. Compared to laser diodes, superluminescent diodes significantly reduce speckle noise artifacts while preserving the sharpness of in-focus regions in the reconstructed images.</figcaption></figure>

We present a wearable full-color holographic augmented reality headset with binocular vision support and real-time embedded hologram calculation. Contrarily to most previously proposed prototypes, our headset employs high-speed amplitude-only microdisplays and embeds a compact and lightweight electronic board to drive and synchronize the microdisplays and light source engines, allowing full-color holographic images to be displayed at 60 Hz. In addition, to enable a standalone usage of the headset, we developed a real-time hologram rendering engine capable of computing holograms at over 35 frames per second on a NVIDIA Jetson AGX Orin embedded platform. Finally, we provide a comparison of the efficiency of laser diodes and superluminescent diodes for the reduction of speckle noise, which greatly affects the reconstructed image's quality. Experimental results show that our prototype enables full-color holographic images to be reconstructed with accurate focus cues and reduced speckle noise in real-time.

CCS Concepts: • Hardware → Emerging technologies.

Additional Key Words and Phrases: holography, near-eye displays, augmented reality, computational displays

**ACM Reference Format:**

Antonin Gilles, Pierre Le Gargasson, Grégory Hocquet, and Patrick Gioia. 2023. Holographic near-eye display with real-time embedded rendering. In *SIGGRAPH Asia 2023 Conference Papers (SA Conference Papers '23), December 12-15, 2023, Sydney, NSW, Australia*. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3610548.3618179

# 1 INTRODUCTION

Augmented reality (AR) has drawn considerable attention from the scientific community during the last decade [Billinghurst et al. 2015]. Indeed, with the recent advances in Head-Mounted Display (HMD) devices, AR may find application in many fields, including manufacturing [Nee et al. 2012], architecture [Chi et al. 2013], medicine [Barsom et al. 2016] and education [Antonioli et al. 2014].

Unfortunately, conventional near-eye glasses – such as Microsoft HoloLens 2® – are based on Stereoscopy, which fails to create a natural and realistic depth illusion. Indeed, Stereoscopy does not reproduce all the Human Visual System (HVS) depth cues perceived in natural vision. In particular, it cannot provide the accommodation stimulus because the viewer has to focus on a fixed focal plane whose depth does not match the actual location of perceived objects. As a consequence, it is often impossible to focus on real and virtual objects simultaneously, which significantly complicates natural interaction with virtual content. Moreover, it creates the Vergence-Accommodation Conflict (VAC) provoking eye-strain and headaches [Hoffman et al. 2008]. To solve these limitations, several alternative three-dimensional (3D) display technologies have been proposed for AR glasses, including varifocal [Akşit et al. 2017], multifocal [Zabels et al. 2019], focal surfaces [Matsuda et al. 2017], light field [Ueno and Takaki 2018] and holographic displays [Chang et al. 2020]. Among these techniques, Holography is often considered as the most promising, since it provides all HVS depth cues without causing eye-strain [Blanche 2021]. To create the depth illusion, a hologram diffracts an illuminating light beam to give it the shape of the light wave that would be emitted, transmitted or reflected by a given scene. As a consequence, the viewers perceive the scene as if it were physically present in front of them.

Thanks to its attractive 3D visualization features, several holographic AR display prototypes were proposed in the last few years. An in-depth review of these works can be found in [Chang et al.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
*SA Conference Papers '23, December 12-15, 2023, Sydney, NSW, Australia*
© 2023 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-0315-7/23/12.
https://doi.org/10.1145/3610548.3618179
2020; Park and Lee 2022]. However, a truly wearable full-color and binocular holographic near-eye display prototype with real-time embedded hologram calculation has never been proposed in the literature. Indeed, as detailed in Section 2, state-of-the-art prototypes still suffer from several challenges that need to be tackled to build such a device. These limitations include the reproduction of full-color images with binocular vision support, the overall system miniaturisation, as well as the development of an embedded hardware calculation engine for real-time hologram rendering.

While solutions have been proposed in the literature to tackle each of these challenges independently, in this paper we demonstrate the non-trivial combination of these components into the first fully integrated and wearable holographic AR headset with embedded hardware hologram calculation.

## 2 STATE-OF-THE-ART CHALLENGES

In the following, we describe the state-of-the-art challenges we tackled to design our holographic near-eye display prototype.

**Reproduction of color images.** Displaying full-color holographic images on compact headsets is still a challenging topic, and most published prototypes were therefore monochromatic [Gao et al. 2017; Murakami et al. 2017; Park and Kim 2018]. A common approach is to synchronize a single phase-only Spatial Light Modulator (SLM) with three red, green and blue light sources to illuminate the color channels sequentially [Choi et al. 2022; Jang et al. 2018; Lee et al. 2022; Maimone et al. 2017; Zhang et al. 2019]. However, although phase holograms have a better diffraction efficiency than their amplitude-only counterparts, they require time-consuming optimization algorithms to achieve an equivalent level of imaging quality, preventing their real-time embedded hardware calculation.

To overcome this limitation, in this paper we employ amplitude-only LCoS microdisplays of resolution (2048 × 1024), capable of displaying holograms with flicker-free color reproduction at 60 Hz. To assess their suitability for holographic near-eye displays, we quantitatively compare their imaging quality and energy efficiency with numerically reconstructed phase-only holograms in Section 5.

**Wearable near-eye display design.** Another issue of holographic near-eye displays is their overall system size and weight. Indeed, since the development of holographic near-eye displays is still an ongoing research topic, it is not surprising that most published works were limited to bulky and heavy benchtop prototypes [Duan et al. 2020; Park and Kim 2018]. Several compact headsets were developed using a miniaturized or folded optical path design to reduce the system size and weight [Gao et al. 2017; Gopakumar et al. 2021; Jang et al. 2022, 2018; Kim et al. 2022, 2018; Maimone et al. 2017; Murakami et al. 2017; Yoneyama et al. 2018; Zhang et al. 2019]. However, none of them embedded the light source and microdisplay driving electronics, which is required for wearing them as HMDs.

In this work, we overcome this engineering challenge by designing a compact and lightweight electronic board to process the input holographic video stream, to drive and synchronize both microdisplays and light source engines as well as to measure and send the interpupillary distance to the hologram rendering engine. This electronic board is directly embedded onto the headset.

**Embedded hardware hologram calculation.** To enable a standalone usage of the headset, holograms must be computed in real-time using compact embedded calculation hardware, which is still very challenging due to the large amount of data to process. In [Yamamoto et al. 2019], the authors developed a compact holographic computer using a Xilinx Zynq UltraScale+ MPSoC with an ARM CPU and a FPGA on a single chip. They were able to compute full-HD monochrome holograms at 15 Hz for a scene containing 6500 points. In [Kim et al. 2019], a layer-based method was implemented on a FPGA to compute full-HD color holograms at 15 Hz. In [Shi et al. 2022] the authors were able to compute full-HD color holograms at 5 Hz on an iPhone 13 Pro using a deep-learning based approach. Finally, in [Gilles 2021], the author used a NVIDIA Jetson AGX Xavier embedded platform to compute a monocular full-color hologram of resolution (2048 × 1024) at more than 50 Hz with an object-based pre-computed angular spectrum approach [Gilles and Gioia 2020]. Nevertheless, the calculation should be further accelerated for our binocular headset prototype, which requires two holograms for the left and right eyes of the viewer to be computed in real time.

To overcome this challenge, we propose a real-time hologram rendering engine capable of computing binocular color holograms at over 35 frames per second on a NVIDIA Jetson AGX Orin embedded platform, described in Section 4.

**Speckle noise reduction.** Another limitation of holographic displays is the presence of significant speckle noise in the reconstructions, which strongly degrades their visual quality. Since speckle noise is due to the high degree of coherence of lasers used to illuminate the hologram, several authors proposed to use partially coherent light sources. In particular, a study published in [Deng and Chu 2017] demonstrated that while the speckle contrast depends on the temporal coherence of the light source, the image sharpness is directly proportional to its spatial coherence. As a consequence, the authors showed that superluminescent diodes (SLEDs), having high spatial and low temporal coherence values, were ideal for holographic display applications. This result was confirmed on holograms of still scenes using a benchtop opaque virtual reality (VR) setup in [Peng et al. 2021], demonstrating that SLEDs are able to efficiently reduce speckle noise artifacts compared to coherent lasers while improving the image sharpness over LEDs.

In this paper, we experimentally validate that the image quality enhancement brought by SLEDs is still clearly noticeable in the case of dynamic holographic videos, even when using a compact and wearable see-through AR headset. To the best of our knowledge, this has never been demonstrated before.

**Field-of-view and eye-box size.** Finally, one of the most important challenges of holographic near-eye displays is to provide a large field-of-view and eye-box size to the viewer. Indeed, as detailed in the supplementary document, the field-of-view is restricted by the SLM resolution and cannot be increased without shrinking the eye-box. Consequently, our headset provides a small field-of-view and eye-box size that do not really support any pupil movement, thus requiring six adjustment wheels to fit the position of the user's eyes. As further discussed in Section 6, SLMs with a much higher resolution are necessary to provide a field-of-view and eye-box size as large as conventional stereoscopic AR displays.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Proposed holographic near-eye display prototype. The mechanical structure of the headset comprises a head support (1), a camera (2) to track the position of the user, as well as six adjustment wheels to modify the interpupillary distance (4) and set the position of the optical system in depth (5) and height (6). To measure the interpupillary distance, a potentiometer (3) is mounted on the beamsplitter combiners. For each eye of the user, the optical system includes an amplitude-only LCoS microdisplay (7), a light source module (8), a beam expander (9), as well as a relay optics system (10). Finally, a compact electronic board (11) is embedded onto the headset to drive and synchronize the microdisplays with the light source modules for color reproduction.</figcaption></figure>

## 3 HEAD-MOUNTED DISPLAY OVERVIEW

Figure 2 shows our proposed holographic head-mounted display prototype, which comprises a mechanical structure, an optical system and a compact electronic board.

**Mechanical structure.** The mechanical elements of the headset were designed using the Autodesk Fusion 360 software and manufactured by a Zortrax M300 Plus 3D printer with black Z-HIPS plastic filament. They enable the electronic board and optical components to be held on the head support, and include six adjustment wheels to modify the interpupillary distance and set the position of the optical system in depth and height. To measure the interpupillary distance, a potentiometer is mounted on the beamsplitter combiners. When the adjustment wheels are operated, the potentiometer moves accordingly and its value is sent to the hologram rendering engine through a USB-C port to update the calculation parameters. A camera is also embedded onto the headset to track the position and orientation of the user. Overall, the headset occupies a total volume of 222 mm × 282 mm × 323 mm and weighs 2.1 kg.

**Optical system.** To support binocular vision, the optical systems corresponding to the left and right eyes of the viewer are constructed symmetrically using off-the-shelf components held with aluminum lens tubes of one inch diameter. Figure 3 shows the optical path of the right eye, which includes a RaonTech RDP551F LCoS microdisplay of resolution ($N_x \times N_y$) = (2048 × 1024) with a pixel pitch of $p = 6.3 \ \mu m$ and a refresh rate of 180 Hz, a red, green and blue (RGB) light source module, a beam expander to enlarge the illuminating light beam diameter, as well as a relay optics system to remove the zero diffraction order and twin image artifacts and increase the field-of-view. To evaluate and compare their impact on the imaging quality of full-color holograms of dynamic 3D scenes, we use a laser diode (LD) and a SLED RGB light source modules purchased from

Exalos. While these two modules have the same red, green and blue wavelengths of 636 nm, 510 nm and 455 nm, respectively, LDs have a much higher temporal coherence than SLEDs, whose bandwidth full width at half maximum (FWHM) is more than five times larger. Overall, the horizontal and vertical field-of-view are given by

$$ \phi_x^{\text{red}} = 18.66^{\circ} \qquad \phi_x^{\text{green}} = 14.96^{\circ} \qquad \phi_x^{\text{blue}} = 13.34^{\circ} \\ \phi_y^{\text{red}} = 9.33^{\circ} \qquad \phi_y^{\text{green}} = 7.48^{\circ} \qquad \phi_y^{\text{blue}} = 6.67^{\circ} $$

for an eye-box size of ($w_x \times w_y$) = (4 mm × 2 mm). It must be noted that the field-of-view in the vertical direction is divided by two compared to the horizontal direction because the spatial filter removes the lower half of the hologram Fourier spectrum to suppress the zero diffraction order and twin image artifacts [Bryngdahl and Lohmann 1968]. The implementation details of the optical system are provided in the supplementary document.

**Electronic board.** Figure 4 shows the functional architecture of the compact electronic board (11), which includes a Mini DisplayPort (mDP) input, an USB-C input/output port, a front-end controller, two microdisplay controllers and a Microcontroller Unit (MCU). All the electronic components are powered by the embedded hologram rendering hardware through the USB-C port with a supply voltage of 5 V and a maximum current of 632 mA. First, the holograms computed by the hologram rendering engine are sent to the front-end controller through the mDP input as a color and binocular video stream of resolution (4096 × 1024). The front-end controller splits the holograms corresponding to the left and right eyes of the viewer into two video streams of resolution (2048 × 1024) which are sent to the microdisplay controllers. These modules sequentially plot the red, green and blue hologram channels onto the microdisplays at a frame rate of 180 Hz and drive the light source modules accordingly for time-multiplexing color reproduction.
<figure><img src="image_4.png" /><figcaption>Fig. 3. Proposed holographic near-eye display optical system (right eye).</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Fig. 4. Functional architecture of the electronic board.</figcaption></figure>

## 4 EMBEDDED HOLOGRAM RENDERING ENGINE

In this section, we describe the proposed embedded hologram rendering engine. To compute the holograms corresponding to the left and right eyes of the user, we use a layer-based hologram synthesis method which comprises four steps, detailed in the following.

**2D-plus-depth rendering.** The first step of our proposed method is to capture the 3D scene geometry from the user's viewpoint as a set of depth layers parallel to the hologram plane. To track the position and orientation of the user with respect to the scene, we used the fiducial marker pipeline of the open-source SolAR framework [Duong et al. 2022]. This pipeline detects the fiducial marker image in the real-time video stream captured by the camera embedded onto the headset and estimates the pose of the camera in relation to the coordinate system of the marker.

Then, 2D-plus-depth perspective projections of the scene are synthesized from the positions of the observer's eyes using two virtual cameras separated by the measured interpupillary distance. Each virtual camera has a resolution of ($N_x \times N_y$) and a horizontal and vertical field-of-view of $\phi_x^{red}$, $\phi_x^{green}$, $\phi_x^{blue}$ for the red, green and blue channels, respectively. As stated previously, the perceived field-of-view in the vertical direction is divided by two compared to the horizontal direction because of the relay optics spatial filter. As a consequence, the lower halves of the color and depth images I and D captured by each camera are rendered with black pixels. Since the depth maps are encoded as 8-bits gray level images, the scene geometry is thus naturally sampled as a set of 256 evenly distributed parallel depth layers.

**Depth map clustering.** To accelerate the hologram calculation with a minimal decrease of visual quality, the second step of our method aims at reducing the number of depth layers. To this end,

the depth map values of each virtual camera are clustered into a set of $N_z \le 256$ depth intervals [$d_i; d_{i+1}$, with $i \in \{0, ..., N_z - 1\}$]. These intervals are computed using the k-means clustering algorithm, which is described in the following.

First, the depth map histogram $\mathcal{H}_D$ is computed and the depth intervals are initialized as

$$ \begin{cases} d_0 = 0, & d_{N_z} = 256 \\ d_{i+1} = \min \left( \left\{ d \left| \sum_{n=d_i}^{d} \mathcal{H}_D[n] > \left\lfloor \frac{N_x N_y}{N_z} \right\rfloor \right\} \right) & i = 0, ..., N_z - 2 \end{cases} \quad (1) $$

where $[a]$ is the smallest integer greater than or equal to a. The depth clusters are then progressively refined through a sequence of assignment and update steps. During the assignment step, the depth intervals are modified such that

$$ d_{i+1} = \min \left( \left\{ d \mid |d - \bar{d}_i| > |d - \bar{d}_{i+1}| \right\} \right) \quad \text{for } i = 0, ..., N_z - 1 \quad (2) $$

where $\bar{d}_i$ is the average depth value of each cluster i, which is then re-computed during the update step. The assignment and update steps are repeated until the clusters no longer change.

Finally, for each depth interval, a layer i is defined to operate as a surface source of light emitting a complex wave $U_i$ sampled on a regular grid of resolution ($N_x \times N_y$), such that

$$ U_i(x, y; c) = \begin{cases} \sqrt{I(x, y; c)} \exp(j\phi(x, y)) & \text{if } D(x, y) \in [d_i; d_{i+1}] \\ 0 & \text{otherwise,} \end{cases} \quad (3) $$

where c is the color channel and $\phi(x, y) \in [0; 2\pi]$ is the initial phase, set to a uniform random value to render a diffuse scene. This layer
is located at depth

$$z_i = \frac{\overline{d_i}}{255} (z_{\max} - z_{\min}) + z_{\min}, \quad (4)$$

where $z_{\min}$ and $z_{\max}$ are the minimal and maximal depths of the scene, respectively.

**Relay optics compensation.** Once the depth layers have been initialized, the third step of our method aims at compensating the scene geometry distortion induced by the relay optics system. Indeed, the converging lenses L2 and L3 deform the geometry of the scene reconstructed by the hologram, and the position of each depth layer must be modified accordingly during the hologram calculation to compensate this effect. We call $d_2$ the distance between the microdisplay and L2, $d_3$ the distance between L2 and L3, $f_2$ and $f_3$ the focal lengths of L2 and L3, respectively. According to the thin lens equation, to compensate the distortion induced by L2 and L3, the depth of layer i should be modified to

$$\hat{z}_i = \frac{f_2(d_3 - \hat{z}_i')}{d_3 - \hat{z}_i' - f_2} - d_2, \quad (5)$$

where

$$\hat{z}_i' = \frac{f_3(e - z_i)}{e - z_i - f_3} \quad (6)$$

is the intermediate compensated layer depth and e is the exit pupil position, given by

$$e = - \frac{((d_2 + d_3)f_2 - d_2d_3)f_3}{(f_2 - d_2)f_3 - (d_2 + d_3)f_2 + d_2d_3}. \quad (7)$$

**Layer-based diffraction.** In the last step of our method, the left and right holograms are computed from their corresponding depth layers. To this end, the light waves scattered by the compensated depth layers are numerically propagated towards the hologram plane and summed up to get the complex-valued object wave

$$O(x, y; c) = \sum_{i=0}^{N_z-1} \mathcal{P}_{\hat{z}_i} \{U_i\} (x, y; c), \quad (8)$$

where $\mathcal{P}_{\hat{z}_i}$ stands for the Fresnel diffraction of light between two parallel planes separated by a distance $\hat{z}_i$, given by

$$\mathcal{P}_{\hat{z}_i} \{U_i\} = \frac{e^{j \frac{2\pi}{\lambda_c} \hat{z}_i}}{j \lambda_c \hat{z}_i} e^{j \frac{\pi}{\lambda_c \hat{z}_i} (x^2 + y^2)} \mathcal{F} \left\{ U_i(\xi, \eta; c) e^{j \frac{\pi}{\lambda_c \hat{z}_i} (\xi^2 + \eta^2)} \right\}. \quad (9)$$

In Eq. (9), $\lambda_c$ is the wavelength of color channel c and $\mathcal{F}$ is the forward Fourier transform. The amplitude-only hologram $H$ is finally obtained by shifting, scaling and quantizing the real part of $O$ on $L = 256$ discrete levels, such that

$$H(x, y; c) = Q(\mathfrak{R} \{O(x, y; c)\}, L, \Delta), \quad (10)$$

where $\mathfrak{R}\{O\}$ stands for the real part of $O$, normalized between -1 and +1, $Q$ is a uniform mid-rise quantizer, given by

$$Q(X, L, \Delta) = \begin{cases} 0 & \text{if } X < -\Delta, \\ \left\lfloor \frac{(X + \Delta)L}{2\Delta} \right\rfloor & \text{if } -\Delta \le X < \Delta, \\ L - 1 & \text{otherwise.} \end{cases} \quad (11)$$

and $\Delta$ refers to the input quantizer range clipping value, which controls the balance between the granular and overload distortions.

**Embedded hardware implementation.** To enable a standalone usage of the headset, we implemented the proposed hologram synthesis method on an NVIDIA Jetson AGX Orin embedded platform, which was released in 2022 as a successor to the Jetson AGX Xavier. It comprises a 12-core ARM Cortex-A78AE v8.2 64-bit CPU, a 2048-core Ampere GPU and 32 GB of DDR5 memory on a single chip of size 100 mm × 87 mm. While having five times less CUDA cores than most recent desktop GPUs such as the NVIDIA 3080Ti, its compact size is perfectly suitable for head-mounted display applications. To facilitate the application deployment, we used the development kit operating Ubuntu 20.04 LTS 64-bit.

In our implementation, the 2D-plus-depth perspective projections of the scene are synthesized using the OpenGL 4.6 API, and all the hologram calculation steps are performed on the GPU using the CUDA application programming interface. To compensate for the reduced number of CUDA cores compared to a desktop GPU, the inner products in Eq. (9) are computed using 16-bit precision operations with one CUDA thread per sample. The Fourier transforms corresponding to the three color channels are then computed simultaneously using the CUDA cuFFT library with batched transforms to accelerate the calculation.

## 5 EXPERIMENTAL RESULTS

For the experiments, we used three input synthetic scenes: Animals, containing three animated models of a fox, a rabbit and a bear, Circuit, representing a race track with a snowy peak in the background, and Village, depicting a cartoon settlement of wooden mountain chalets. While Animals is representative of AR scenes showcasing distinct and isolated 3D models, Circuit and Village are typical VR scenes spanning a continuous and large depth range.

**Calculation time.** To evaluate the performance of our proposed hologram synthesis method in terms of calculation time, we implemented it on two different platforms: the NVIDIA Jetson AGX Orin embedded platform and a desktop computer employing an Intel Core i9-9900X CPU operating at 3.50 GHz, a main memory of 32 GB, as well as a GPU NVIDIA GeForce RTX 2080Ti.

Figure 5 shows the average calculation times and frame rates for the computation of binocular and colorful holograms of resolution (2048 × 1024) depending on the number of depth layers $N_z$. We can see that for both implementations the calculation time is proportional to the number of depth layers. Using the desktop computer, the calculation time ranges between 4.7 ms and 16.6 ms for one to eight layers, corresponding to frame rates between 212.8 Hz and 60.2 Hz. On the other hand, since the NVIDIA Jetson AGX Orin contains less than half the number of CUDA cores than the NVIDIA GeForce RTX 2080Ti, the embedded hardware platform is slower than the desktop computer. For a single layer, the binocular hologram calculation frame rate is 73.5 Hz, which is higher than the microdisplays refresh rate. For two and three layers, the holographic video stream is computed at a frame rate of 48.1 Hz and 35.6 Hz, respectively, which is still sufficient to provide a real-time AR experience without flickering. However, between four and eight layers, the calculation frame rate drops under 28.2 Hz, creating flickering in the reconstructions. For this reason, in the following we use only three depth layers.
<figure><img src="image_6.png" /><figcaption>Fig. 5. Calculation times and frame rates for the computation of binocular and colorful holograms of resolution (2048 × 1024) using the proposed method, depending on the number of depth layers $N_z$.</figcaption></figure>

**Power consumption.** To measure the instantaneous power consumption of the headset prototype and embedded hologram rendering engine, we used the `jtop` command of the `jetson-stats` monitoring package [Bonghi 2022]. This tool reported a total CPU, GPU and headset power consumption of 12 W, regardless of the number of layers. Consequently, with a Lithium-Ion battery having a capacity of 10000 mAh and a voltage of 3.7 V, our headset can be used autonomously for more than three hours. Overall, these experimental results demonstrate that the proposed embedded hologram rendering engine can be used in holographic AR applications.

**Imaging quality and energy efficiency assessment.** To assess the suitability of amplitude holograms for near-eye displays, we compared their imaging quality and energy efficiency with phase-only counterparts computed from the complex-valued object wave given in Eq. (8) using three different methods:

* the amplitude discard method, in which only the phase of the object wave is kept and quantized on 256 levels,

* the double phase method described in [Maimone et al. 2017], where each object wave sample is encoded using two adjacent phase values quantized on 256 levels, and

* the binary phase method, in which the object wave amplitude is discarded and its phase is quantized on two levels.

Iterative phase optimization algorithms were not considered in this study because their high computational burden is incompatible with real-time embedded hardware implementations.

To avoid any bias from the differences in pixel pitches and resolutions or by the imperfections of the optical system, we numerically simulated their optical reconstructions on SLMs with a resolution of (2048 × 1024) and a pixel pitch of $p = 6.3 \ \mu m$. However, for double phase holograms we simulated a SLM with a resolution of (2048 × 2048) and non-squared pixels of size ($p \times p/2$). In addition, to simulate their time-multiplexing display on binary SLMs having a refresh rate of 4.5 kHz per channel (such as the M150 from Forth Dimension Displays), we computed 25 binary phase holograms for each scene using different initial random phase distributions in Eq. (3) and averaged their numerical reconstructions. It must be noted that we simulated the necessary spatial filters – *i.e.* a single-sideband filter for the amplitude-only and binary phase and a pupil aperture for double phase holograms – in the reconstructions.

To evaluate the imaging quality of amplitude and phase-only holograms, we computed the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) [Wang et al. 2004] of their simulated optical reconstructions with respect to those obtained from the complex-valued object wave given in Eq. (8). While the PSNR estimates the absolute error between two numerically reconstructed scene images, the SSIM is a measure of the structural similarity between two images that takes into account their luminance, contrast and structure and assigns a similarity index between 0 and 1, where 1 indicates identical images.

Figures 6a and 6b show the PSNR and SSIM of the numerically reconstructed amplitude holograms computed from *Circuit* with three depth layers, depending on the input quantizer range clipping value $\Delta$ used in Eq. (11). For the sake of comparison, the corresponding metrics of phase-only holograms are plotted in dashed lines. As shown in these figures, the PSNR of amplitude-only holograms increases with $\Delta$ to reach a threshold value of 40.2 dB for $\Delta = 0.5$. From $\Delta = 0.16$, it exceeds the PSNR of phase holograms, which are of 25.2, 27.6, and 27.7 dB for the binary phase, double phase and amplitude discard methods, respectively. Similarly, the SSIM of amplitude holograms increases with $\Delta$ to reach a threshold value of 0.97 for $\Delta = 0.5$. Regardless of $\Delta$, it exceeds the SSIM of phase holograms, which are of 0.30, 0.40, and 0.62 for the binary phase, double phase and amplitude discard methods, respectively. Interestingly, while the amplitude discard and double phase methods outperform binary phase holograms with respect to PSNR, the latter exhibits superior SSIM performance. This is due to the fact that although the two-level quantization of binary phase holograms induces a significant error on each pixel of the reconstructed image, the temporal averaging process reduces speckle noise compared to other methods and thus better preserves the structure of the reconstructed images.

Following these results, it is tempting to use the largest $\Delta$ value to obtain the best imaging quality. However, as shown in Figure 6c, the energy efficiency – already very low for amplitude holograms – decreases even further as $\Delta$ increases, dropping to 1.4% when $\Delta = 0.5$. For comparison, the corresponding double phase hologram energy efficiency is 21.6% due to the spatial filtering by the used pupil aperture, and those of amplitude discard and binary phase holograms are always of 100% and 50% because of the single-sideband filter, respectively. In practice, we found that a value of $\Delta = 0.4$ offers a good compromise between energy efficiency and image quality. This value is used in the rest of the experiments.

These metrics are confirmed by the numerical reconstructions shown in Figure 7. Indeed, while the binary phase hologram reconstruction exhibits less speckle noise thanks to the temporal averaging process, it presents a non-zero background noise and a significantly reduced image contrast compared to the reconstructed complex-valued object wave. The amplitude discard and double phase holograms exhibit a similar behavior, with additional speckle noise. Overall, the numerically reconstructed amplitude hologram with $\Delta = 0.4$ is the closest to the reference complex-valued object wave, confirming the superior imaging quality of amplitude-only holograms over phase holograms in terms of contrast and speckle noise. Further details on this imaging quality and energy efficiency assessment study are given in the supplementary document, together with experimental results for the *Animals* and *Village* scenes.
**Optical reconstructions.** To demonstrate the real-time hologram calculation pipeline from 3D pose estimation to display, we used a virtual scene containing a single animated fox model whose position was set a few centimeters above the fiducial marker image. Figure 8 and the supplementary video show the photographs captured through our prototype using the LD module, with a varying fiducial marker position and orientation. As shown in this video, the location of the fox is updated in real-time with a small latency by the embedded hologram rendering engine.

To evaluate the reproduction accuracy of the scene geometry and accommodation depth cue, we used the *Animals* scene with a fixed user position. The animated fox, rabbit and bear models were located at a distance of 50 cm, 150 cm and 300 cm away from the user's eyes, respectively, and three real wooden toys – an acrobat, a ring of fire and a tamer – were placed at the same distances in front of the headset. Figure 9 and the supplementary video show the optical reconstructions of holograms illuminated by the LD and SLED modules, recorded with a camera focusing on different depths. For both light sources, when the camera is focusing on a given 3D model, the corresponding wooden toy is also focused, demonstrating that our prototype accurately provides the accommodation depth cue. In addition, the scene geometry is properly reproduced without distortion thanks to the compensation step of our hologram synthesis method. Finally, compared to LDs, the SLED module significantly reduce speckle noise artifacts while preserving the sharpness of in-focus regions in the reconstructed images. For both light sources, the power passing through the exit pupil was measured to be under 0.2 mW, which is compliant with Class 1 laser safety conditions.

To assess the imaging quality of holograms computed from VR scenes *Circuit* and *Village*, we replaced the non-polarizing beam-splitter combiner by a mirror. As shown in Figure 10, when the scene spans a continuous depth range, the separation between consecutive depth layers is visible in the optical reconstructions, creating visual artifacts at the layers' boundaries. Therefore, while the layer-based calculation approach is suitable for typical AR content containing only isolated 3D models, it is not ideal for VR scenes with a continuous depth extent.

# 6 OPEN CHALLENGES

While our prototype enables full-color holographic images to be reconstructed with reduced speckle noise in real-time, it still suffers from several open challenges which are discussed below.

**Headset size and weight.** First of all, although the designed compact electronic board and embedded hologram rendering engine enable our prototype to be worn and used autonomously, its large form factor and weight are incompatible with a comfortable prolonged use. This is due to the fact that we only used off-the-shelf optical and optomechanical components to build the optical system. In particular, the prototype's weight and size come largely from the aluminum lens tubes and mounts used to hold and align the optical components. Indeed, these elements are made of aluminum, unlike the mechanical structure of the headset, which was made of plastic. In a future work, we plan to reduce the prototype's weight and size by designing and manufacturing a miniaturized optical system with custom optical and optomechanical components.

**Holographic imaging quality.** Secondly, although SLEDs enable to reduce speckle noise, our prototype does not reach the imaging quality of consumer stereoscopic near-eye displays. This is mostly due to optical components misalignment as well as geometric and chromatic aberrations. Indeed, since the optical system was built using off-the-shelf components, it was not optimized to reduce these aberrations. In a future work, we plan to improve the imaging quality by designing and manufacturing custom lenses to optimize the optical system in reducing these aberrations.

**Field-of-view and eye-box size.** Finally, since it cannot be increased without shrinking the eye-box, the field-of-view of our headset is still too narrow for many practical consumer applications. To solve this trade-off, several consumer AR headsets use eye-box replication waveguides to obtain a wide field-of-view without vignetting [Kress 2020]. However, these waveguides can only be used if the input field is collimated, in other words when the image is formed in the far field. Unfortunately, since holographic images are formed in the near field, each replicated exit pupil will produce an image at a slightly different distance and the same image will be overlapped with different focal depths. For this reason, these waveguides cannot be used in the context of holographic displays.

Consequently, to provide a field-of-view of (53° × 26.5°) with an eye-box of (4 mm × 2 mm), SLMs of resolution (8192 × 4096) are required, resulting in slower hologram calculation frame rates. Alternatively, a promising solution is to use a beam steering element to deviate the illuminating light beam and shift the exit pupil to different locations sequentially depending on the user's gaze, as proposed in [Jang et al. 2018]. Nevertheless, this requires a precise and low latency eye-tracking system to follow the short and rapid eye movements called saccades, reaching up to 700°/s.

To conclude, while state-of-the-art approaches can be used to improve the headset size, weight and imaging quality, the limited field-of-view of holographic near-eye displays is still an open challenge that poses a significant obstacle to their practical usability.

# 7 CONCLUSION

In this paper, we built the first fully integrated and wearable holographic AR headset with embedded hardware hologram calculation capable of full-color reproduction with binocular vision support. Our proposed prototype embeds all the optical components and driving electronics on a single head support and employs amplitude-only LCoS microdisplays instead of the commonly used phase SLM. In addition, we proposed and implemented a real-time layer-based hologram calculation method capable of computing binocular color holograms at over 35 frames per second on a NVIDIA Jetson AGX Orin embedded platform, enabling its standalone usage.

Experimental results show that our prototype enables full-color holographic images to be reconstructed with accurate focus cues in real-time. Thanks to the SLED illumination, speckle noise artifacts are reduced, enhancing the imaging quality of the headset. In a future work, we plan to reduce the prototype's weight and size and improve its imaging quality by designing a miniaturized optical system with custom optical and optomechanical components. In addition, the limited field-of-view of holographic near-eye displays is still an open challenge that requires further research to be solved.
# ACKNOWLEDGMENTS

This work has been funded by the French government through the National Research Agency (ANR) Investment referenced ANR-A0-AIRT-07. All the 3D models used in this paper were downloaded from the Unity Asset Store (https://assetstore.unity.com).

# REFERENCES

Kaan Akşit, Ward Lopes, Jonghyun Kim, Peter Shirley, and David Luebke. 2017. Near-eye Varifocal Augmented Reality Display Using See-through Screens. *ACM Trans. Graph*. 36, 6 (Nov. 2017), 189:1–189:13. https://doi.org/10.1145/3130800.3130892

Misty Antonioli, Corinne Blake, and Kelly Sparks. 2014. Augmented Reality Applications in Education. *The Journal of Technology Studies* 40, 1/2 (2014), 96–107. https://www.jstor.org/stable/43604312 Publisher: Epsilon Pi Tau, Inc.

E. Z. Barsom, M. Graafland, and M. P. Schijven. 2016. Systematic review on the effectiveness of augmented reality applications in medical training. *Surgical Endoscopy* 30, 10 (Oct. 2016), 4174–4183. https://doi.org/10.1007/s00464-016-4800-6

Mark Billinghamurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. *Foundations and Trends in Human-Computer Interaction* 8, 2-3 (March 2015), 73–272. https://doi.org/10.1561/1100000049 Publisher: Now Publishers, Inc.

Pierre-Alexandre Blanche. 2021. Holography, and the future of 3D display. *Light: Advanced Manufacturing* 2, 4 (Dec. 2021), 446–459. https://doi.org/10.37188/lam.2021.028 Publisher: Light: Advanced Manufacturing.

Raffaello Bonghi. 2022. Jetson stats. https://github.com/rbonghi/jetson_stats original-date: 2018-11-24T19:42:07Z.

Olof Bryngdahl and Adolf Lohmann. 1968. Single-Sideband Holography. *JOSA* 58, 5 (May 1968), 620–624. https://doi.org/10.1364/JOSA.58.000620

Chenliang Chang, Kiseung Bang, Gordon Wetzstein, Byoungho Lee, and Liang Gao. 2020. Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective. *Optica* 7, 11 (Nov. 2020), 1563–1578. https://doi.org/10.1364/OPTICA.406004 Publisher: Optica Publishing Group.

Hung-Lin Chi, Shih-Chung Kang, and Xiangyu Wang. 2013. Research trends and opportunities of augmented reality applications in architecture, engineering, and construction. *Automation in Construction* 33 (Aug. 2013), 116–122. https://doi.org/10.1016/j.autcon.2012.12.017

Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, Matthew O'Toole, and Gordon Wetzstein. 2022. Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators. In *ACM SIGGRAPH 2022 Conference Proceedings (SIGGRAPH '22)*. Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3528233.3530734

Yuanbo Deng and Daping Chu. 2017. Coherence properties of different light sources and their effect on the image sharpness and speckle of holographic displays. *Scientific Reports* 7, 1 (July 2017), 5893. https://doi.org/10.1038/s41598-017-06215-x

Xinhui Duan, Juan Liu, Xueliang Shi, Zhiqi Zhang, and Jiasheng Xiao. 2020. Full-color see-through near-eye holographic display with 80° field of view and an expanded eye-box. *Optics Express* 28, 21 (Oct. 2020), 31316–31329. https://doi.org/10.1364/OE.399359 Publisher: Optical Society of America.

Nam-Duong Duong, Christophe Cutullic, Jean-Marie Henaff, and Jérôme Royan. 2022. AR Cloud: Towards Collaborative Augmented Reality at a Large-Scale. In *2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)*. 733–738. https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00155 ISSN: 2771-1110.

Qiankun Gao, Juan Liu, Xinhui Duan, Tao Zhao, Xin Li, and Peilin Liu. 2017. Compact see-through 3D head-mounted display based on wavefront modulation with holographic grating filter. *Optics Express* 25, 7 (April 2017), 8412–8424. https://doi.org/10.1364/OE.25.008412

Antonin Gilles. 2021. Real-time embedded hologram calculation for augmented reality glasses. In *2021 International Conference on Visual Communications and Image Processing (VCIP)*. IEEE, 1–5. https://doi.org/10.1109/VCIP53242.2021.9675435 ISSN: 2642-9357.

Antonin Gilles and Patrick Gioia. 2020. Real-time computer-generated hologram calculation using pre-computed angular spectra. In *Optics, Photonics and Digital Technologies for Imaging Applications VI*, Vol. 11353. International Society for Optics and Photonics, 1135304. https://doi.org/10.1117/12.2554537

Manu Gopakumar, Jonghyun Kim, Suyeon Choi, Yifan Peng, and Gordon Wetzstein. 2021. Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays. *Optics Letters* 46, 23 (Dec. 2021), 5822–5825. https://doi.org/10.1364/OL.442851 Publisher: Optica Publishing Group.

David M. Hoffman, Ahna R. Girshick, Kurt Akeley, and Martin S. Banks. 2008. Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. *Journal of Vision* 8, 3 (March 2008), 33. https://doi.org/10.1167/8.3.33

Changwon Jang, Kiseung Bang, Minseok Chae, Byoungho Lee, and Douglas Lanman. 2022. Waveguide Holography: Towards True 3D Holographic Glasses. https://doi.org/10.48550/arXiv.2211.02784 arXiv:2211.02784 [physics].

Changwon Jang, Kiseung Bang, Gang Li, and Byoungho Lee. 2018. Holographic Near-eye Display with Expanded Eye-box. *ACM Trans. Graph*. 37, 6 (Dec. 2018), 195:1–195:14. https://doi.org/10.1145/3272127.3275069

Hojung Kim, Yongkyu Kim, Hyunwook Ji, Hyunsik Park, Jungkwuen An, Hoon Song, Yun Tae Kim, Hong-Seok Lee, and Kichul Kim. 2019. A Single-Chip FPGA Holographic Video Processor. *IEEE Transactions on Industrial Electronics* 66, 3 (March 2019), 2066–2073. https://doi.org/10.1109/TIE.2018.2835424 Conference Name: IEEE Transactions on Industrial Electronics.

Jonghyun Kim, Manu Gopakumar, Suyeon Choi, Yifan Peng, Ward Lopes, and Gordon Wetzstein. 2022. Holographic Glasses for Virtual Reality. In *ACM SIGGRAPH 2022 Conference Proceedings (SIGGRAPH '22)*. Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3528233.3530739

Mugeon Kim, Sungjin Lim, Geunseop Choi, Youngmin Kim, Hwi Kim, and Joonku Hahn. 2018. Expanded Exit-Pupil Holographic Head-Mounted Display With High-Speed Digital Micromirror Device. *ETRI Journal* 40, 3 (June 2018), 366–375. https://doi.org/10.4218/etrij.2017-0166

Bernard C. Kress. 2020. *Optical Architectures for Augmented-, Virtual-, and Mixed-Reality Headsets*. SPIE. https://doi.org/10.1117/3.2559304

Byounghyo Lee, Dongyeon Kim, Seungjae Lee, Chun Chen, and Byoungho Lee. 2022. High-contrast, speckle-free, true 3D holography via binary CGH optimization. *Scientific Reports* 12, 1 (Feb. 2022), 2811. https://doi.org/10.1038/s41598-022-06405-2 Number: 1 Publisher: Nature Publishing Group.

Andrew Maimone, Andreas Georgiou, and Joel S. Kollin. 2017. Holographic near-eye Displays for Virtual and Augmented Reality. *ACM Trans. Graph*. 36, 4 (July 2017), 85:1–85:16. https://doi.org/10.1145/3072959.3073624

Nathan Matsuda, Alexander Fix, and Douglas Lanman. 2017. Focal Surface Displays. *ACM Trans. Graph*. 36, 4 (July 2017), 86:1–86:14. https://doi.org/10.1145/3072959.3073590

Eishin Murakami, Yuki Oguro, and Yuji Sakamoto. 2017. Study on Compact Head-Mounted Display System Using Electro-Holography for Augmented Reality. *IEICE Transactions on Electronics* E100.C (Nov. 2017), 965–971. https://doi.org/10.1587/transele.E100.C.965

A. Y. C. Nee, S. K. Ong, G. Chrysolouris, and D. Mourtzis. 2012. Augmented reality applications in design and manufacturing. *CIRP Annals* 61, 2 (Jan. 2012), 657–679. https://doi.org/10.1016/j.cirp.2012.05.010

Jae-Hyeung Park and Seong-Bok Kim. 2018. Optical see-through holographic near-eye display with eyebox steering and depth of field control. *Optics Express* 26, 21 (Oct. 2018), 27076–27088. https://doi.org/10.1364/OE.26.027076

Jae-Hyeung Park and Byoungho Lee. 2022. Holographic techniques for augmented reality and virtual reality near-eye displays. *Light: Advanced Manufacturing* 3, 1 (Jan. 2022), 1–14. https://doi.org/10.37188/lam.2022.009 Publisher: Light: Advanced Manufacturing.

Yifan Peng, Suyeon Choi, Jonghyun Kim, and Gordon Wetzstein. 2021. Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration. *Science Advances* 7, 46 (Nov. 2021), eabg5040. https://doi.org/10.1126/sciadv.abg5040 Publisher: American Association for the Advancement of Science.

Liang Shi, Beichen Li, and Wojciech Matusik. 2022. End-to-end learning of 3D phase-only holograms for holographic display. *Light: Science & Applications* 11, 1 (Aug. 2022), 247. https://doi.org/10.1038/s41377-022-00894-6 Number: 1 Publisher: Nature Publishing Group.

Takaaki Ueno and Yasuhiro Takaki. 2018. Super multi-view near-eye display to solve vergence-accommodation conflict. *Optics Express* 26, 23 (Nov. 2018), 30730–30715. https://doi.org/10.1364/OE.26.030703

Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. *IEEE Transactions on Image Processing* 13, 4 (April 2004), 600–612. https://doi.org/10.1109/TIP.2003.819861 Conference Name: IEEE Transactions on Image Processing.

Yota Yamamoto, Nobuyuki Masuda, Ryuji Hirayama, Hirotaka Nakayama, Takashi Kakue, Tomoyoshi Shimobaba, and Tomoyoshi Ito. 2019. Special-purpose computer for electroholography in embedded systems. *OSA Continuum* 2, 4 (April 2019), 1166–1173. https://doi.org/10.1364/OSAC.2.001166 Publisher: Optical Society of America.

Takuo Yoneyama, Eishin Murakami, Yuki Oguro, Hibiki Kubo, Kazuhiro Yamaguchi, and Yuji Sakamoto. 2018. Holographic head-mounted display with correct accommodation and vergence stimuli. *Optical Engineering* 57, 6 (May 2018), 061619. https://doi.org/10.1117/1.OE.57.6.061619

Roberts Zabels, Krïss Osmanis, Mărtinș Narels, Ugis Gertners, Ainărs Ozols, Kārlis Rūtenbergs, and Ilmārs Osmanis. 2019. AR Displays: Next-Generation Technologies to Solve the Vergence–Accommodation Conflict. *Applied Sciences* 9, 15 (Jan. 2019), 3147. https://doi.org/10.3390/app9153147 Number: 15 Publisher: Multidisciplinary Digital Publishing Institute.

Zhiqi Zhang, Juan Liu, Qiankun Gao, Xinhui Duan, and Xueliang Shi. 2019. A full-color compact 3D see-through near-eye display system based on complex amplitude modulation. *Optics Express* 27, 5 (March 2019), 7023–7035. https://doi.org/10.1364/OE.27.007023 Publisher: Optical Society of America.
<figure><img src="image_7.png" /><figcaption>Fig. 6. Objective quality assessment and energy efficiency metrics of the numerically reconstructed amplitude holograms computed from *Circuit* with three depth layers, depending on the input quantizer range clipping value $\Delta$ defined in Eq. (11). The PSNR and SSIM metrics were computed with respect to the reference complex-valued object wave given in Eq. (8). For the sake of comparison, the corresponding phase-only holograms metrics are plotted in dashed lines.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 7. Numerically reconstructed holograms computed from *Circuit* with three depth layers, focused on the foreground. Red squares indicate the position of zoom-in details shown in the upper-right corner of each reconstruction.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 8. Photographs captured through our prototype using the LD module, demonstrating the real-time hologram calculation pipeline from 3D pose estimation to display. The fox position is accurately updated by the hologram rendering engine in real-time and with a very small latency depending on the fiducial marker position and orientation (see supplementary video).</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>Fig. 9. Photographs captured through our prototype with a camera focusing on different depths, using the LD (top row) and SLED modules (bottom row). When the camera is focusing on a given 3D model, the corresponding real-world object is also focused while other parts of the scene appear blurred, demonstrating that our proposed holographic AR headset provides accurate accommodation depth cues. Compared to LDs, the SLED module significantly reduce speckle noise artifacts while preserving the sharpness of in-focus regions in the reconstructed images (see supplementary video).</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 10. Photographs captured through our prototype in virtual reality mode with a camera focusing on different depths, for two scenes covering a large depth extent: *Circuit* (first row) and *Village* (second row). When scene objects span a continuous depth range, the separation between consecutive depth layers is visible, creating visual artifacts at the layers boundaries. As a consequence, while the layer-based calculation approach is suitable for typical AR content containing only isolated 3D models, it is not ideal for VR scenes with a continuous depth extent.</figcaption></figure>
# Holographic near-eye display with real-time embedded rendering

Supplementary document

ANTONIN GILLES and PIERRE LE GARGASSON, Institute of Research & Technology b<>com, France
GRÉGORY HOCQUET and PATRICK GIOIA, Orange Labs, France

This supplementary document describes the implementation of the proposed holographic augmented reality headset optical system, and provides additional details and experimental results on the imaging quality and energy efficiency assessment study.

## ACM Reference Format:

Antonin Gilles, Pierre Le Gargasson, Grégory Hocquet, and Patrick Gioia. 2023. Holographic near-eye display with real-time embedded rendering: Supplementary document. In *SIGGRAPH Asia 2023 Conference Papers (SA Conference Papers '23), December 12–15, 2023, Sydney, NSW, Australia*. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3610548.3618179

## S1 PROPOSED OPTICAL SYSTEM

In the following, we describe the optical system of the proposed holographic head-mounted display. Figure 1 shows the optical path corresponding to the right eye, which includes:

* a RGB light source module, described in Section S1.1,

* a beam expander made of a diverging lens L1 and a converging lens L2 to enlarge the illuminating light beam diameter, described in Section S1.2,

* a polarizing beamsplitter and an amplitude-only LCoS microdisplay to display the computed holograms,

* a relay optics system comprising two converging lenses L2 and L3 as well as a spatial filter to remove the zero diffraction order and twin image artifacts, described in Section S1.3,

* and a non-polarizing beamsplitter to superimpose the reconstructed virtual image on the real environment.

### S1.1 Light source

To enable light diffraction, the holograms must be illuminated by a spatially and temporally coherent light source. While laser diodes (LD) are commonly used in this context, a study published in [Deng and Chu 2017] showed that superluminescent diodes (SLED) are ideal for holographic display applications: thanks to their reduced temporal coherence, they bring less speckle noise in the reconstructed images. To evaluate and compare their impact on the imaging quality of full-color holograms of dynamic 3D scenes, we use two different RGB light sources purchased from Exalos: a LD and a SLED modules, whose characteristics are presented in Table 1.

As shown in this table, while these two modules share the same center wavelengths, LDs have a much higher temporal coherence than SLEDs, whose bandwidth full width at half maximum (FWHM)

<figure><img src="image_12.png" /><figcaption>Fig. 1. Optical system of the right eye.</figcaption></figure>

is more than five times larger. In addition, their maximum output power of 10 mW per color is sufficiently high to compensate for the energy loss due to the low diffraction efficiency of amplitude-only modulation. Finally, while the red, green and blue light beams are divergent, they are perfectly aligned at the output of the modules, allowing the use of a single optical path for the three colors.

### S1.2 Beam expander

The objective of the beam expander is to enlarge the illuminating light beam at the output of the light source module to illuminate the entire surface of the microdisplay. Its schematic is shown in Figure 2. In this figure, L1 and L2 are two lenses of focal length $f_1$ and $f_2$, respectively, separated by a distance $d_1$. The incident beam on L1, located at a distance $d_0$ from the light source, has diameter $D_{in}$ and a full divergence angle $\alpha$. At the output of the beam expander, the beam diameter $D_{out}$ must be larger than the microdisplay size $S$.

<table><caption>Table 1. Light source modules characteristics.</caption><thead><tr><th>Color</th><th>Wavelength</th><th colspan="2">Bandwidth FWHM</th><th>Power</th></tr></thead><tbody><tr><td>Red</td><td>636 nm</td><td>LD: &lt;1 nm</td><td>SLED: 6 nm</td><td>10 mW</td></tr><tr><td>Green</td><td>510 nm</td><td>LD: &lt;1 nm</td><td>SLED: 10 nm</td><td>10 mW</td></tr><tr><td>Blue</td><td>455 nm</td><td>LD: &lt;1 nm</td><td>SLED: 5 nm</td><td>10 mW</td></tr></tbody></table>

<footnote>Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).</footnote>

<footnote>SA Conference Papers '23, December 12–15, 2023, Sydney, NSW, Australia</footnote>

<footnote>© 2023 Copyright held by the owner/author(s).</footnote>

<footnote>ACM ISBN 979-8-4007-0315-7/23/12.</footnote>

<footnote>https://doi.org/10.1145/3610548.3618179</footnote>
<figure><img src="image_13.png" /><figcaption>Fig. 2. Beam expander system.</figcaption></figure>

To build the beam expander, we use a diverging lens L1 of focal length $f_1 = -6$ mm located at a distance $d_0 = 15$ mm from the light source module, and a converging lens L2 of focal length $f_2 = 150$ mm. The light beam diameter incident on L1 is measured to be $D_{in} = 0.42$ mm, with a full divergence angle of $\alpha = 0.02$ rad. In order to produce a collimated output beam, $d_1$ must be set to

$$d_1 = \frac{(\alpha f_1 - D_{in})f_2 - D_{in}f_1}{\alpha f_1 - D_{in}} = 145.33 \text{ mm.} \quad (1)$$

At the output of the beam expander, the beam diameter $D_{out}$ is therefore given by

$$D_{out} = \frac{(\alpha d_1 + D_{in})f_1 - D_{in}d_1}{f_1} = 13.5 \text{ mm.} \quad (2)$$

In our prototype, we use two RaonTech RDP551F LCOS microdisplays with a resolution of $(N_x \times N_y) = (2048 \times 1024)$ and a pixel pitch of $p = 6.3 \ \mu\text{m}$ to display the holograms corresponding to the left and right eyes of the viewer. The surface to illuminate is therefore given by $(S_x \times S_y) = (12.9 \ \text{mm} \times 6.45 \ \text{mm})$, which is smaller than the output beam diameter $D_{out}$.

## S1.3 Relay optics

Figure 3 shows the schematics of the relay optics system, whose objective is to filter out the zero diffraction order and twin image artifacts and to increase the field-of-view $\phi$. In this figure, L2 and L3 are two converging lenses of focal length $f_2$ and $f_3$, respectively, separated by a distance $d_3$. To reduce the optical path size, the L2 lens, located at a distance $d_2$ from the microdisplay, is shared by the beam expander and relay optics systems.

In addition, a spatial filter is located in the focal plane of L2 to remove the unwanted diffraction orders using the single-sideband technique [Bryngdahl and Lohmann 1968]. In this technique, only the object light rays that are propagating downwards – i.e. the higher half of the hologram Fourier spectrum – are computed, by rendering the lower halves of the color and depth images captured by each camera with black pixels. Then, during the reconstruction step, the spatial filter removes the zero-order and twin-image artifacts by cutting off the lower half of the hologram Fourier spectrum. For the sake of clarity, this filter is not shown in Figure 3.

The exit pupil position, also called eye relief, is given by

$$e = \frac{((d_2 + d_3)f_2 - d_2d_3)f_3}{(f_2 - d_2)f_3 - (d_2 + d_3)f_2 + d_2d_3}. \quad (3)$$

<figure><img src="image_14.png" /><figcaption>Fig. 3. Relay optics system.</figcaption></figure>

At this position, the perceived field-of-view $\phi_x$ and $\phi_y$ in the horizontal and vertical directions, respectively, are given by

$$\phi_x = - \frac{((f_2 - d_2)f_3 - (d_2 + d_3)f_2 + d_2d_3)\theta}{f_2f_3} \quad \text{and} \quad \phi_y = \frac{\phi_x}{2}, \quad (4)$$

where $\theta$ is twice the maximum diffraction angle, given by

$$\theta = 2 \arcsin \left( \frac{\lambda}{2p} \right), \quad (5)$$

and $p$ is the microdisplay pixel pitch. The field-of-view in the vertical direction is divided by two compared to the horizontal direction because of the single-sideband spatial filter.

As shown in Eq. (4), the field-of-view can be widened by using a smaller focal length $f_3$ or by increasing $d_3$. However, these parameters have an impact on the exit pupil size, also called eye-box, which defines the region within which the whole image can be viewed without vignetting. Indeed, the eye-box in the horizontal and vertical directions is given by

$$\begin{pmatrix} w_x \\ w_y \end{pmatrix} = - \frac{f_2 f_3}{(f_2 - d_2) f_3 - (d_2 + d_3) f_2 + d_2 d_3} \begin{pmatrix} S_x \\ S_y \end{pmatrix}. \quad (6)$$

As a consequence, when $f_3$ decreases or $d_3$ increases, the eye-box is shrunk, creating vignetting when the eyes of the user move. This trade-off between the eye-box size and field-of-view also exists in the case of conventional stereoscopic AR displays. In practice, an eye-box size of around 4 mm is often used in the literature [Murakami et al. 2017; Yoneyama et al. 2018].

In our holographic AR headset prototype, we use a converging lens L2 of focal length $f_2 = 150$ mm located at a distance $d_2 = 10$ mm from the microdisplay and a converging lens L3 of focal length $f_3 = 50$ mm located at a distance $d_3 = 212$ mm from L2. The horizontal and vertical field-of-view are therefore given by

$$\phi_x^{\text{red}} = 18.66^{\circ}, \quad \phi_x^{\text{green}} = 14.96^{\circ}, \quad \phi_x^{\text{blue}} = 13.34^{\circ} \quad (7)$$

$$\phi_y^{\text{red}} = 9.33^{\circ}, \quad \phi_y^{\text{green}} = 7.48^{\circ}, \quad \phi_y^{\text{blue}} = 6.67^{\circ} \quad (8)$$

for an eye-box size of $(w_x \times w_y) = (4 \ \text{mm} \times 2 \ \text{mm})$.

## S1.4 Detailed components list

In the following are listed the components required for assembling the proposed optical system, numbered in reference to Figure 1.

(1) Light source modules: Exalos RGB LD and SLED butterfly modules with free-space output

(2) Lens L1: Thorlabs biconcave lens LD2746-A

(3) Lens L2: Thorlabs achromatic doublet AC254-150-A-ML
(4) Polarizing beamsplitter: Thorlabs CCM1-PBS251/M

(5) Microdisplays: RaonTech RDP551F LCoS panel

(6) Spatial filter: custom 3D printed component

(7) Mirror: Thorlabs dielectric turning mirror CCM1-E02/M

(8) Lens L3: Thorlabs achromatic doublet AC254-050-A-ML

(9) Non-polarizing beamsplitter: Thorlabs CCM1-BS013/M

These components are held and aligned together using Thorlabs aluminium lens tubes of one inch diameter.

# S2 IMAGING QUALITY AND ENERGY EFFICIENCY ASSESSMENT STUDY

In the following, we provide further details on the imaging quality and energy efficiency assessment study, together with additional experimental results.

## S2.1 Compared hologram calculation methods

To assess the suitability of amplitude-only holograms for near-eye displays, we compared their imaging quality and energy efficiency with phase-only counterparts computed from the complex-valued object wave using three different methods: the amplitude discard, double phase and binary phase methods. In the following, we recall how the object wave and amplitude-only holograms are computed by the proposed embedded rendering engine, and provide additional details on the compared phase-only hologram calculation methods.

**Complex-valued object wave.** In our proposed embedded hologram rendering engine, the 3D scene geometry is reconstructed from the synthesized 2D-plus-depth images as a sparse set of parallel depth layers whose distance $\hat{z}_i$ from the hologram plane is set as to compensate the distortion induced by the relay optics system described in Section S1.3. Then, the light waves scattered by the depth layers are numerically propagated towards the hologram plane and summed up to get the complex-valued object wave

$$O(x, y; c) = \sum_{i=0}^{N_z-1} \mathcal{P}_{\hat{z}_i} \{U_i\} (x, y; c), \quad (9)$$

where c is the color channel index, $N_z$ is the number of depth layers, $U_i$ is the light wave scattered by layer i, and $\mathcal{P}_{\hat{z}_i}$ is the Fresnel diffraction of light between two parallel planes, given by

$$\mathcal{P}_{\hat{z}_i} \{U_i\} = \frac{e^{j \frac{2\pi}{\lambda_c} \hat{z}_i}}{j \lambda_c \hat{z}_i} e^{j \frac{\pi}{\lambda_c \hat{z}_i} (x^2 + y^2)} \mathcal{F} \{U_i(\xi, \eta; c) e^{j \frac{\pi}{\lambda_c \hat{z}_i} (\xi^2 + \eta^2)}\}. \quad (10)$$

In Eq. (10), $\lambda_c$ is the wavelength of color channel c and $\mathcal{F}$ is the forward Fourier transform. Since the 2D-plus-depth images of the scene are synthesized from the position of the observer's eyes with perspective projection pinhole cameras, the occlusions are naturally taken into account in the complex-valued object wave for these specific viewpoints only.

**Amplitude-only holograms.** Once the object wave $O$ has been computed, the amplitude-only hologram $H$ is obtained by shifting, scaling and quantizing the real part of $O$ on $L = 256$ discrete levels, such that

$$H(x, y; c) = Q(\mathcal{R} \{O(x, y; c)\}, L, \Delta), \quad (11)$$

where $\mathcal{R}\{O\}$ stands for the real part of $O$, normalized between -1 and +1, $Q$ is a uniform mid-rise quantizer, given by

$$Q(X, L, \Delta) = \begin{cases} 0 & \text{if } X < -\Delta, \\ \left[ \frac{(X + \Delta)L}{2\Delta} \right] & \text{if } -\Delta \le X < \Delta, \\ L - 1 & \text{otherwise.} \end{cases} \quad (12)$$

and $\Delta$ refers to the input quantizer range clipping value, which controls the balance between the granular and overload distortions.

Since they do not modulate the phase of the incident light wave, amplitude-only holograms produce the unwanted zero diffraction order and twin image artifacts. To remove them, we use a single-sideband filter in the focal plane of L2, as detailed in Section S1.3.

**Amplitude discard method.** The amplitude discard method is the most trivial way of computing a phase-only hologram from the object wave. In this approach, only the phase of $O$ is kept and quantized on $L = 256$ discrete levels, such that

$$H(x, y; c) = \left[ \arg \{O(x, y; c)\} \frac{L}{2\pi} \right], \quad (13)$$

where $\arg\{O\}$ is the phase angle of $O$. Since they do not produce any unwanted diffraction order artifact, phase-only holograms computed using the amplitude discard method do not need any spatial filter in the optical system.

**Double phase method.** The double phase method [Hsueh and Sawchuk 1978] is an alternative approach for generating phase-only holograms which involves encoding both the amplitude and phase information of the object wave into two separate phase values. Indeed, each sample $O(x, y; c)$ can be represented as the sum of two complex numbers with constant amplitude and variable phase, such that

$$O(x, y; c) = 0.5 \left( e^{j\phi_1(x, y; c)} + e^{j\phi_2(x, y; c)} \right), \quad (14)$$

$$\phi_1(x, y; c) = \arg \{O(x, y; c)\} - \cos^{-1}(|O(x, y; c)|), \quad (15)$$

$$\phi_2(x, y; c) = \arg \{O(x, y; c)\} + \cos^{-1}(|O(x, y; c)|), \quad (16)$$

where $|O|$ is the amplitude of $O$, normalized between 0 and 1.

In our experiments, we simulated the double phase encoding arrangement described in [Maimone et al. 2017], where the pair of phase values $\phi_1$ and $\phi_2$ are quantized on $L = 256$ discrete levels and placed adjacently in a checkerboard pattern on the Spatial Light Modulator (SLM). Since this arrangement implicitly creates a high frequency grating, the unwanted diffraction orders are simply removed by using a pupil aperture in the focal plane of L2.

**Binary phase method.** Finally, another approach to compute phase-only holograms is the binary phase method, in which the object wave amplitude is discarded and its phase is quantized on two levels, such that

$$H(x, y; c) = \begin{cases} 1 & \text{if } \arg \{O(x, y; c)\} > \pi, \\ 0 & \text{otherwise.} \end{cases} \quad (17)$$

This method enables to take advantage of the high display frame rates of binary phase modulation SLMs such as Ferroelectric LCoS (FLCoS) [Lee et al. 2022] or Digital Micromirror Devices (DMD) [Choi
et al. 2022]. For instance, the M150 FLCOS SLM from Forth Dimension Displays is able to display binary monochromatic images at 4.5 kHz, i.e. 25 times faster than the RaonTech RDP551F LCoS panel used in our prototype. By displaying several holograms computed with different initial random phase diffusers sequentially, the speckle noise can be strongly reduced thanks to time multiplexing.

Nevertheless, the binary quantization process produces the unwanted zero diffraction order and twin image artifacts, which need to be removed using a single-sideband filter in the focal plane of L2 as for amplitude-only holograms.

## S2.2 Simulation of the optical reconstructions

To compare the imaging quality and energy efficiency of amplitude and phase-only holograms without any bias from the optical system's imperfections, we numerically simulated their reconstructions on SLMs with a resolution of (2048 × 1024) and a pixel pitch of $p = 6.3 \ \mu m$. However, for double phase holograms, which require twice the number of pixels, we simulated a SLM with a resolution of (2048 × 2048) and non-squared pixels of size ($p \times p/2$).

The numerical reconstruction process is shown in Figure 4. For a focus distance of $\hat{z}_i$, the simulated reconstructed image intensity from the object wave $O$ is calculated as

$$I_i(x, y; c) = |\mathcal{P}_{\hat{z}_i}^{-1}\{O\}(x, y; c)|^2, \quad (18)$$

where $\mathcal{P}_{\hat{z}_i}^{-1}$ is the inverse Fresnel diffraction, given by

$$\mathcal{P}_{\hat{z}_i}^{-1}\{O\} = \frac{j\lambda_c \hat{z}_i}{e^{j\frac{2\pi}{\lambda_c} \hat{z}_i}} e^{-j\frac{\pi}{\lambda_c \hat{z}_i} (x^2+y^2)} \mathcal{F}^{-1}\{U_i(\xi, \eta; c)e^{-j\frac{\pi}{\lambda_c \hat{z}_i}(\xi^2+\eta^2)}\}. \quad (19)$$

The corresponding numerically reconstructed image intensity of amplitude and phase-only holograms is computed as

$$\hat{I}_i(x, y; c) = |\mathcal{P}_{\hat{z}_i}^{-1}\{\mathcal{G}\{\hat{O}\}\}(x, y; c)|^2, \quad (20)$$

where $\hat{O}$ is the complex-valued light wave in the hologram plane and $\mathcal{G}$ is the optional filtering operation to remove the unwanted diffraction artifacts. For amplitude holograms, $\hat{O}$ is set to

$$\hat{O}(x, y; c) = \frac{4\Delta}{L} H(x, y; c). \quad (21)$$

Note that the light wave amplitude is scaled by $\frac{4\Delta}{L}$ instead of $\frac{2\Delta}{L}$ to compensate for the light energy loss due to the single-sideband filter. For phase holograms, $\hat{O}$ is given by

$$\hat{O}(x, y; c) = A_c \exp\left(\frac{2j\pi}{L} H(x, y; c)\right), \quad (22)$$

where $A_c$ is the constant amplitude of channel $c$, whose value is set for each hologram such as to maximize the Peak-Signal-To-Noise Ratio (PSNR) of the numerically reconstructed image intensity.

For amplitude-only, binary phase and double phase holograms, the unwanted diffraction artifacts are removed by a spatial filter in the focal plane of L2. This filtering operation is simulated by

$$\mathcal{G}\{\hat{O}\} = \mathcal{F}^{-1}\{\mathcal{F}\{\hat{O}\}(\xi, \eta)G(\xi, \eta)\}, \quad (23)$$

where $G$ is the spatial filter. For amplitude-only and binary phase holograms, $G$ is a single-sideband filter, given by

$$G(\xi, \eta) = \begin{cases} 1 & \text{if } \eta > 0, \\ 0 & \text{otherwise.} \end{cases} \quad (24)$$

For double phase holograms, $G$ is a pupil aperture modeled as

$$G(\xi, \eta) = \begin{cases} 1 & \text{if } |\eta| < \frac{1}{2p}, \\ 0 & \text{otherwise.} \end{cases} \quad (25)$$

Since holograms computed using the amplitude discard method do not need any spatial filter, $\mathcal{G}$ is set to the identity function.

Finally, to simulate their time-multiplexing display on DMDs or FLCOS having a refresh rate of 4.5 kHz per channel (such as the M150 SLM from Forth Dimension Displays), we computed 25 binary phase holograms for each scene using different initial random phase diffusers and averaged their numerical reconstructions.

## S2.3 Imaging quality and energy efficiency assessment

To evaluate the imaging quality of amplitude and phase-only holograms, we computed the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) [Wang et al. 2004] of their simulated optical reconstructions with respect to those obtained from the complex-valued object wave. While the PSNR estimates the absolute error between two numerically reconstructed scene images, the SSIM is a measure of the structural similarity between two images that takes into account their luminance, contrast and structure and assigns a similarity index between 0 and 1, where 1 indicates identical images. To compute these metrics, $\hat{I}_i$ and $I_i$ are first scaled with respect to the maximum value of $I_i$, and quantized on 256 levels. Then, the PSNR and SSIM metrics are computed for each reconstruction depth $i$ independently and their arithmetic mean is calculated.

To evaluate their energy efficiency $EE$, we computed the ratio between the illuminating and reconstructed light waves intensities over the hologram area, such that

$$EE = \frac{\int_x \int_y |\mathcal{G}\{\hat{O}\}(x, y)|^2 dxdy}{\int_x \int_y |R|^2 dxdy}, \quad (26)$$

where $R$ is the illuminating light wave amplitude, given by $R = \frac{4\Delta}{L}$ and $R = A_c$ for amplitude and phase-only holograms, respectively. While the energy efficiency of phase-only holograms computed using the amplitude discard method is always equal to 100%, those of binary phase holograms is equal to 50% because of the single-sideband filter used in the numerical reconstructions. Finally, the energy efficiencies of amplitude and double-phase holograms depend on the scene content.

## S2.4 Experimental results

Figures 5, 6 and 7 show the image quality and energy efficiency metrics of the numerically reconstructed amplitude holograms computed from *Animals*, *Circuit* and *Village* with three depth layers, depending on the input quantizer range clipping value $\Delta$ used in Eq. (12). For the sake of comparison, the corresponding metrics of phase-only holograms are plotted in dashed lines. As shown in these
<figure><img src="image_15.png" /><figcaption>Fig. 4. Numerical simulation of the optical reconstruction process for the imaging quality and energy efficiency assessment.</figcaption></figure>

figures, the PSNR and SSIM of amplitude-only holograms increase with $\Delta$ to reach a threshold value around $\Delta = 0.5$, and exceed those of phase holograms when $\Delta > 0.16$. Interestingly, while the amplitude discard and double phase methods outperform binary phase holograms with respect to PSNR, the latter exhibits superior SSIM performance. This is due to the fact that although the two-level quantization of binary phase holograms induces a significant error on each pixel of the reconstructed image, the temporal averaging process reduces speckle noise compared to other methods and thus better preserves the structure of the reconstructed images.

Following these results, it is tempting to use the largest $\Delta$ value to obtain the best imaging quality. However, the energy efficiency – already very low for amplitude holograms – decreases even further as $\Delta$ increases, dropping around 1.4% when $\Delta = 0.5$. For comparison, the corresponding double phase hologram energy efficiency are of 27.6%, 21.6% and 29.7% for the *Animals*, *Circuit* and *Village*, respectively. In practice, we found that a value of $\Delta = 0.4$ offers a good compromise between energy efficiency and image quality.

These experimental results are confirmed by the numerical reconstructions shown in Figures 8, 9 and 10. Indeed, we can see that the binary phase hologram reconstructions exhibit significantly reduced speckle noise artifacts compared to the amplitude discard and double phase holograms, at the cost of background noise. Overall, the numerically reconstructed amplitude holograms with $\Delta = 0.4$ are always the closest to the reconstructions obtained with the reference complex-valued object waves, and present the best imaging quality in terms of contrast and speckle noise.

## ACKNOWLEDGMENTS

This work has been funded by the French government through the National Research Agency (ANR) Investment referenced ANR-A0-AIRT-07. All the 3D models used in this paper were downloaded from the Unity Asset Store (https://assetstore.unity.com).

## REFERENCES

* Olof Bryngdahl and Adolf Lohmann. 1968. Single-Sideband Holography. JOSA 58, 5 (May 1968), 620–624. https://doi.org/10.1364/JOSA.58.000620

* Suyeon Choi, Manu Gopakumar, Yifan Peng, Jonghyun Kim, Matthew O'Toole, and Gordon Wetzstein. 2022. Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators. In *ACM SIGGRAPH 2022 Conference Proceedings (SIGGRAPH '22)*. Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3528233.3530734

* Yuanbo Deng and Daping Chu. 2017. Coherence properties of different light sources and their effect on the image sharpness and speckle of holographic displays. *Scientific Reports* 7, 1 (July 2017), 5893. https://doi.org/10.1038/s41598-017-06215-x

* C. K. Hsueh and A. A. Sawchuk. 1978. Computer-generated double-phase holograms. *Applied Optics* 17, 24 (Dec. 1978), 3874–3883. https://doi.org/10.1364/AO.17.003874

* Byounghyo Lee, Dongyeon Kim, Seungjae Lee, Chun Chen, and Byoungho Lee. 2022. High-contrast, speckle-free, true 3D holography via binary CGH optimization. *Scientific Reports* 12, 1 (Feb. 2022), 2811. https://doi.org/10.1038/s41598-022-06405-2

* Andrew Maimone, Andreas Georgiou, and Joel S. Kollin. 2017. Holographic Near-eye Displays for Virtual and Augmented Reality. *ACM Trans. Graph.* 36, 4 (July 2017), 85:1–85:16. https://doi.org/10.1145/3072959.3073624

* Eishin Murakami, Yuki Oguro, and Yuji Sakamoto. 2017. Study on Compact Head-Mounted Display System Using Electro-Holography for Augmented Reality. *IEICE Transactions on Electronics* E100.C (Nov. 2017), 965–971. https://doi.org/10.1587/ transele.E100.C.965

* Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. *IEEE Transactions on Image Processing* 13, 4 (April 2004), 600–612. https://doi.org/10.1109/TIP.2003.819861

* Takuo Yoneyama, Eishin Murakami, Yuki Oguro, Hibiki Kubo, Kazuhiro Yamaguchi, and Yuji Sakamoto. 2018. Holographic head-mounted display with correct accommodation and vergence stimuli. *Optical Engineering* 57, 6 (May 2018), 061619. https://doi.org/10.1117/1.OE.57.6.061619
<figure><img src="image_16.png" /><figcaption>Fig. 5. Objective quality assessment and energy efficiency metrics of the numerically reconstructed amplitude and phase holograms computed from *Animals* with three depth layers, depending on the input quantizer range clipping value $Δ$.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>Fig. 6. Objective quality assessment and energy efficiency metrics of the numerically reconstructed amplitude and phase holograms computed from *Circuit* with three depth layers, depending on the input quantizer range clipping value $Δ$.</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>Fig. 7. Objective quality assessment and energy efficiency metrics of the numerically reconstructed amplitude and phase holograms computed from *Village* with three depth layers, depending on the input quantizer range clipping value $Δ$.</figcaption></figure>
<figure><img src="image_19.png" /><figcaption>Fig. 8. Numerically reconstructed holograms computed from *Animals* with three depth layers.</figcaption></figure>
<figure><img src="image_20.png" /><figcaption>Fig. 9. Numerically reconstructed holograms computed from *Circuit* with three depth layers.</figcaption></figure>
<figure><img src="image_21.png" /><figcaption>Fig. 10. Numerically reconstructed holograms computed from *Village* with three depth layers.</figcaption></figure>