A comprehensive framework for visual quality assessment of light field tensor displays

In recent years, light field technology has attracted the interest of academia and industry, thanks to the possibility of rendering 3D scenes in a more realistic and immersive way. In particular, light field displays have been consistently investigated for their ability to offer a glass-free 3D viewing experience. Among others, tensor displays represent a promising way to render light field contents. However, only a few prototypes of such type of displays have been implemented and are available to the scientific community. As a direct consequence, the visual quality of such displays has not been rigorously investigated. In this paper, we propose a new framework to assess the visual quality of light field tensor displays on conventional 2D screens. The multilayer components of the tensor displays are virtually rendered on a typical 2D monitor through the use of a GUI, and different viewing angles can be accessed by simple mouse interactions. Both single and double stimulus methodologies for subjective quality assessment of light field contents are supported in this framework, while the total time of interaction is recorded for every stimulus. Results obtained in two different laboratory settings demonstrate that the framework can be successfully used to perform subjective quality assessment of different compression solutions for light field tensor displays. Introduction Light field photography has recently seen a surge of popularity due to the increased capabilities in acquiring and rendering real-life scenes in a more immersive way. In particular, light field rendering promises to overcome the limitations of stereoscopic representation by allowing for a more seamless transition between multiple point of views, thus giving a more faithful representation of 3D scenes. The recent innovations in the realm of acquisition and compression of light field contents have fueled the need for light field displays on which the data can be natively visualized. In the past, several multi-view displays have been presented that allow to visualize the scene from several points of views. Among other technologies, parallax barrier displays have been proposed as a glass-free alternative to stereoscopic displays [8]. To allow for multi-view rendering, several technologies have been implemented, such as polarizer [14] or LCD dynamic barriers through viewer tracking [12]. Other approaches include the use of integral imaging [1], or a system of projectors used in combination with a holographic screen to fully render the 3D scene [2]. The Quality of Experience (QoE) associated with such type of light field displays has been recently investigated by Tamboli et al. [17][18]. Another alternative uses a stack of programmable light-attenuating layers in front of a light-emitting source to provide depth cues without the need of glasses [10][11] [20]. The layer patterns to be displayed in each light-attenuating layers can be obtained from the multi-view light field data through Non-negative Tensor Factorization (NTF) [20]. Recently, a new method has been proposed to generate the layer patterns from a stack of focused images (focal stack), which greatly reduces the number of images that are needed as input for the tensor displays [16]. The method was tested on a prototype 3D display to prove its efficacy [9]. Testing the visual quality of compressed and uncompressed light field contents on native light field display is of extreme importance in future development of both new rendering methods, as well as new compression solutions. However, the limited availability of light field displays hinders the assessment of their visual quality. Moreover, hardware limitations in prototype models considerably lessen the perceptual quality of experience in consuming light field contents. Being able to simulate light field multi-layer rendering in a virtual environment is thus helpful in conducting evaluation of visual quality for light field displays in an ideal scenario. In this paper, we present a framework to conduct quality assessment of light field contents rendered through a tensor display simulator using 2D screens. Through a GUI the layer patterns composing the multi-layer tensor displays are simulated in a 3D environment. By interacting with the mouse, users can experience the light field from different points of views. A validating experiment is conducted to show the validity of our framework. Proposed framework The framework proposed in this paper provides a tool to assess the quality of light field contents rendered through the use of multi-layer tensor displays. It consists of a software application for quality assessment of light field contents that enables visualization from different points of views, while keeping track of both the given ratings and the total time of interaction. A graphical interface based on the software proposed in [15], simulates the multi-layer structure of light field tensor displays. The layer patterns are given as input to be directly visualized using the interface, along with a file specifying the parameters for the rendering, such as the horizontal and vertical angular resolution (i.e., the number of perspective views) of the input light field, and the number of layers composing the simulated display. The Figure 1: Example rendering of the input stimuli with the proposed graphical interface, using double stimulus methodology with side-byside display. (a) Bikes (b) Danger de mort (c) Stone Pillars Outside (d) Fountain & Vincent (e) Friends Figure 2: Central perspective view from each content used in the test. layer patterns are always displayed in their original resolution. By clicking and dragging, users can physically alter the visualization angle of the simulated display on the screen, thus accessing different points of views. The viewing angles are limited by the number of layers and the angular resolutions of the input light field, to ensure only properly rendered points of view are accessible. In particular, denoting Vx and Vy as the number of perspective views of the input light field in the horizontal and vertical dimension, respectively, and L as the number of layers in the simulated display, the maximum viewing angle θx and θy in the horizontal and vertical direction, respectively, can be defined as such: θx = arctan α ⌊ Vx 2 ⌋