nils-herrmann/clean_scirepeval huggingface dataset:
Towards an efficient methodology for evaluation of quality of experience in Augmented Reality

The goal of this paper is to survey existing quality assessment methodologies for Augmented Reality (AR) visualization and to introduce a methodology for subjective quality assessment. Methodologies to assess the quality of AR systems have existed since these technologies appeared. The existing methodologies typically take an approach from the fields they are used in, such as ergonomics, usability, psychophysics or ethnography. Each field utilizes different methods, looking at different aspects of AR quality such as physical limitations, tracking loss or jitter, perceptual issues or feedback issues, just to name a few. AR systems are complex experiences, involving a mix of user interaction, visual perception, audio, haptic or other types of multimodal interactions as well. This paper focuses on the quality assessment of AR visualization, with a special interest on applications for neuronavigation.
Does quality impact the business model? Case: Digital Cinema

The media scene has endured continuous changes ever since we saw the light of the motion picture in 1888 into our present digital age. The changes, innovations and expansions have generated a significant media value chain constituting a vast worldwide economy. The D-Cinema business is about creating a digital media asset (the film, the feature or motion picture) and screening this on as many screens as possible. Hence enabling the studios, owning the asset, optimizing their profit and making sure the exhibitors sell more tickets. For more than 100 years this has been managed by a value chain consisting of the studios, the distributors and the exhibitors (cinema owners). Digitization has the potential of dramatically changing the industry and opens for new business models driven by the innovation and governed by change and the handling of change. This paper explores the innovation driven change in the cinema business going digital and assesses the impact quality has on the roll out and the underlying business model.
Visual Contrast Sensitivity Guided Video Quality Assessment

Contrast sensitivity is an important characteristic of the human visual system (HVS), which is widely used in image and video signal processing. For visual quality assessment, static spatio-temporal frequency based contrast sensitivity function (CSF) is often used, while the contrast sensitivity can also be affected by smooth pursuit of eyes when tracking attentive regions in the field of view. This paper proposes to tune CSF based on an attention map derived from a visual attention model. The tuned CSF formulated by spatial frequency, temporal velocity, and visual attention map is used to filter video signals in order to construct a quality metric according to the difference of the filtered signals between a reference video and its distorted version. Experimental results demonstrate that the proposed attention tuned spatio-velocity CSF outperforms the traditional spatio-temporal CSF in evaluating the perceived video quality.
Concealment aware mode selection for power-rate-distortion optimized H.264/AVC encoder

In our recent work, an analytic power-channe/error-rate-distortion (PERD) model was developed to estimate the end-to-end distortion of power constrained portable devices in real-time video communications over bandwidth limited and lossy channels. A random mode selection scheme is implemented after obtaining optimal percentages of intra, inter and skip mode through a constrained optimization process. In this paper, advanced solution to the problem of mode selection for each macroblock (MB) is proposed. A concealment aware PERD model is first derived, which takes error concealment at the decoder into consideration to estimate the end-to-end distortion. By using this model, optimal percentages of intra, inter and skip modes that minimize decoder distortion are obtained under any power and rate constraints. An intelligent mode selection scheme is then generalized by ranking the priority of MB according to how well it gets reconstructed if lost. Experimental results show that, the new framework consistently achieves better power-rate-distortion (P-R-D) performance with little extra computation load, which is critical in real-time communications of portable devices.
Spatial and temporal pooling of image quality metrics for perceptual video quality assessment on packet loss streams

Video streaming through bandwidth-limited channels often suffer from packet losses. Therefore, perceptual quality assessment on video sequences with packet losses is a critical issue in digital video communications. This paper analyzes several image quality metrics and evaluates their applications using spatial and temporal pooling schemes in perceptual video quality assessment for video streams with packet losses. Several approaches using Minkowski summation and averages over different distorted spatial regions and temporal frames to pool the spatial and temporal qualities are evaluated. The experimental results with respect to the subjective video quality measurements demonstrate that the subjects are more sensitive to the most annoying spatial regions and temporal segments when assessing the video quality of the lossy streams.
A streaming media engine using digital item adaptation

This paper describes the use of capability descriptors for MPEG-21 digital item adaptation. The capability descriptors describe the terminal and network for an application involving media resource delivery. The capability descriptors are used in a streaming media environment through a test bed demonstrating possible functionalities of a universal multimedia access (UMA) enabled system. The test bed involves viewer configuration through capability negotiations. The configuration triggers a media resource adaptation for accessing streaming media using a variety of access schemes on a diverse range of terminals. The application focus is towards future generation mobile and wireless multimedia communications, ensuring the user gets the same reliability and seamlessness as in a fixed environment.
Adaptive H.264/MPEG-4 SVC video over IEEE 802.16 broadband wireless networks

In this paper we present a solution for delivering streaming video-on-demand to subscribers via an 802.16 broadband wireless access network. The solution leverages the forthcoming H.264/AVC Scalable Video Coding (SVC) scheme and a mechanism to perform rate adaptation based on monitoring changes to the amount of flow traffic in the network at any time. A simulation-based approach is used to determine how the system performs in a rural deployment. Results show that the scheme provides high utilization of the wireless access system, at over 96%. Further, it maintains smooth transmission rate for the video applications, and ensures that no interruptions in continuous video playback occur during the streaming session. Lastly, a comparison with single-layer H.264/AVC is performed, showing how the proposed solution performs better with respect to both system utilization and the fact that no clients suffer interruptions in continuous playback.
Stereoscopic quality datasets under various test conditions

Quality assessment on emerging stereoscopic 3D media is still in an early stage when compared to 2D image/video quality assessment. New characteristics (e.g. specific artifact perception, visual strain, perceived depth) and various requisite factors in stereoscopic systems (e.g. system-introduced crosstalk, screen size, viewing position, scene content, camera baseline) should be taken into account. However, most subjective quality assessments on stereoscopic 3D have been focused on coding artifacts and their influence on the perceived viewing experience. Thus, demand on sharing datasets on perceived visual quality under various requisite factors of stereoscopic systems is increasing. In this paper, we publish three visual quality datasets, including crosstalk stereoscopic (relates crosstalk perception to scene content, crosstalk level and camera baseline), crosstalk auto-stereoscopic (relates crosstalk perception to scene content, crosstalk level and viewing position) and QoE stereoscopic (relates QoE to scene content, camera baseline, screen size and viewing position). Moreover, the laboratory environments, visual stimuli, testing methodologies for generating these datasets and results analysis are also detailed in the paper.
Visual attention tuned spatio-velocity contrast sensitivity for video quality assessment

Contrast sensitivity is an important characteristic of the human visual system (HVS), which is widely used in image and video signal processing. For visual quality assessment, static spatio-temporal frequency based contrast sensitivity function (CSF) is often used, while the contrast sensitivity can also be affected by smooth pursuit of eyes when tracking attentive regions in the field of view. This paper proposes to tune CSF based on an attention map derived from a visual attention model. The tuned CSF formulated by spatial frequency, temporal velocity, and visual attention map is used to filter video signals in order to construct a quality metric according to the difference of the filtered signals between a reference video and its distorted version. Experimental results demonstrate that the proposed attention tuned spatio-velocity CSF outperforms the traditional spatio-temporal CSF in evaluating the perceived video quality.
Transmission of still images over noisy channels

This paper presents the importance of error resilience in still image coding in order to be able to transmit images over noisy channels. The motivation for error resilience is described as well as a presentation of the ongoing work on error resilience in the standardising of the next generation still image compression system, JPEG2000. As a case study we focus on some theoretical results obtained comparing the tradeoffs between re-transmission time, bit error rate (BER) and quality for typical mobile and wireless applications, showing the benefits of having an error resilient mode.
Subjective visual quality assessment in the presence of audio for Digital Cinema

In this paper, we investigate whether the presence of audio with different quality levels can influence the outcome of subjective visual quality assessment in a Digital Cinema setting. We asked the participants to judge the visual quality when watching an audiovisual content in a Digital Cinema environment and investigated whether the participants can neglect the presence and the quality of audio. The stimuli used were 10 seconds long color sequences accompany with orchestral music at 2K resolution, 24 fps, and YCbCr 4:4:4 played on DCI certified equipment. The result show that in visual only subjective quality assessment, the presence of audio (low or high quality) does not significantly influence on the visual quality judgment.
MPEG-21: The 21st century multimedia framework

The lack of interoperable (and thus standardized) solutions is stalling the deployment of advanced multimedia packaging and distribution applications although most of the individual technologies are indeed already present. This motivated MPEG (ISO/IEC JTCl SC29 WGI I) in June 2000 to start working on the definition of enabling normative technology for the multimedia applications of the 21st century: MPEG-21 "Multimedia Framework." MPEG-21's approach is to define a framework to support transactions that are interoperable and highly automated, specifically taking into account digital rights management (DRM) requirements and targeting multimedia access and delivery using heterogeneous networks and terminals. This article first outlines the context and background of the MPEG-21 initiative. Then, an overview of MPEG-21 technology is given. Subsequently, it is discussed how MPEG-21 can provide solutions for universal multimedia access (UMA). UMA is also one of the use cases that has led to the creation of a new part in MPEG-21 dealing with digital item adaptation. Finally, this article concludes with an overview of MPEG-21 related activities and an outlook on future developments.
Quality of dirty: A decision making assessment methodology for automatic license plate recognition under dirtied conditions

Automatic license plate recognition (ALPR) from vehicle images plays an important role in intelligent traffic management. A critical problem is that the cameras can be dirtied such that the captured images are unrecognizable. This paper defines a new concept of Quality of Dirty (QoD) of captured images indicating when the dirtied cameras need to be cleaned. An accurate image QoD metric is proposed based on relevant image features and support vector regression (SVR), and it can handle a tricky issue of differentiating captured imaged containing dirtied vehicles from images captured by a dirtied camera. A subjective assessment to collect ground-truth QoD scores of real captured images has also been conducted. Experiments have demonstrated that the proposed image QoD metric achieves high accuracy when predicting the dirtied degree of cameras in the tunnel environment.
Balancing Attended and Global Stimuli in Perceived Video Quality Assessment

The visual attention mechanism plays a key role in the human perception system and it has a significant impact on our assessment of perceived video quality. In spite of receiving less attention from the viewers, unattended stimuli can still contribute to the understanding of the visual content. This paper proposes a quality model based on the late attention selection theory, assuming that the video quality is perceived via two mechanisms: global and local quality assessment. First we model several visual features influencing the visual attention in quality assessment scenarios to derive an attention map using appropriate fusion techniques. The global quality assessment as based on the assumption that viewers allocate their attention equally to the entire visual scene, is modeled by four carefully designed quality features. By employing these same quality features, the local quality model tuned by the attention map considers the degradations on the significantly attended stimuli. To generate the overall video quality score, global and local quality features are combined by a content adaptive linear fusion method and pooled over time, taking the temporal quality variation into consideration. The experimental results have been compared to results from appropriate eye tracking and video quality assessment experiments, demonstrating promising performance.
An objective metric for assessing quality of experience on stereoscopic images

Most quality models for stereoscopic presentations are dedicated to measuring quality degradation caused by compression artefacts. However, non-compression distortions induced during acquisition and presentation usually have significant influence on 3D viewing experience. In this paper, we propose an objective metric for viewing experience assessment by taking camera baseline and binocular distortion crosstalk into consideration. In particular, the proposed metric is based on our previous work on both subjective evaluation and objective assessment of crosstalk perception. Results on a publicly available stereoscopic quality database demonstrate that the proposed metric can achieve more than 87% correlation with subjective assessment of viewing experience.
Quality Incentive Assisted Congestion Control for Receiver-Driven Multicast

The potential problem of oversubscribing receivers in receiver-driven multicast is addressed. We present a framework based on harmonizing the erasure-resilience properties of video with existing congestion control algorithms. The result offers subscription alternatives for receivers in which a penalty in terms of visual quality will be experienced by oversubscribing receivers. Thus, we provide an incentive for performing proper congestion control. The presented framework is independent of specific congestion control algorithms. Simulation results show the intended performance.
A robust, low complexity 5.0 kbps stochastic coder for a noisy satellite channel

A code excited linear predictive coding (CELP) based coder operating at 5065 bps is presented. The coder uses a self-excitation sequence for long-term prediction in addition to a fixed codebook populated with stochastic multipulse vectors as the excitation signal. The authors describe how the complexity of the coder is reduced, and present methods of increasing its spectral robustness, thus making it a suitable candidate for mobile satellite communication. The complexity is greatly reduced by using overlapping codevectors in both the adaptive and multipulse codebook. For robustness a filter correction scheme is derived, which, together with frame repetition techniques, will let the coder operate with little perceivable degradation in burst durations up to 40 ms.<<ETX>>
Spatial immersion versus emotional immersion, which is more immersive?

In this paper, we defined and measured two types of immersion, namely, spatial immersion and emotional immersion, on flat 2D screen displays and using a 33-item questionnaire. Our results show that emotional immersion is significantly more immersive than spatial immersion in terms of sense of “being there”, time perception, realism, sense of engagement, emotional aspects, sensory cues, etc. Spatial immersion is almost as immersive as emotional emotion in terms of attention and image motion, and spatial immersion is more immersive than emotional immersion in terms of spatial dis-orientation. Our results also show that there are individual differences in the perception of immersion. Finally, we linked parameters of measurement with QoE influencing factors, in an aim to bridge the gap in the QoE assessment and modelling of immersive experience in storytelling.
Dynamic Hardware Management of the H264/AVC Encoder Control Structure Using a Framework for System Scenarios

Many modern applications exhibit dynamic behavior, which can be exploited for reduced energy consumption. We employ a two-phase combined design-time/run-time methodology that identifies different run-time situations and clusters similar behaviors into system scenarios. This methodology is integrated with our framework for system scenario based designs, which dynamically tune the hardware to match the application behavior. We achieve significant energy reductions for an extracted control structure of a video codec widely used in hand-held devices today. We encode a video stream consisting of different frame sizes based on measured available wireless bandwidth. Energy consumption is measured with a modified microcontroller board from Atmel with two alternative voltage and frequency settings. While maintaining the perceptual video quality and frame rate, our method results in up to 44% energy reduction for our encoded streams, even after including an average worst-case tuning overhead of 8.5%. In reality this overhead is typically negligible since the worst-case assumes very frequent tuning, while in realistic situations it will be performed at a much lower rate. This makes the expected gain over 50%.
Unequal Error Protection for Embedded Codes over Parallel Packet Erasure Channels

This paper presents extensions to previous algorithms for unequal error protection (UEP) of multimedia encoded using embedded source coders. Whereas earlier work on this topic has considered the single-channel case only, we investigate the case where multiple parallel channels with possibly different loss characteristics are used. Such scenarios are becoming more relevant with the gradual deployment of priority-enabled networks (e.g. DiffServ). We present results indicating that significant performance gains can be achieved by using UEP instead of the simpler EEP (equal error protection) scheme also in the multichannel case. Furthermore, it is seen that the performance gap between UEP and EEP increases with an increasing number of channels
Loss-distortion estimation for practical H.264/AVC streams

Even though several unequal erasure protection (UEP) schemes have been proposed for video streaming, it is still a challenge to define the optimal relative protection level for different data units. In this paper, we study loss-distortion modeling in realistic video streaming scenarios. The main observation is that when video streams with complex hierarchical structures and error resilience features are considered, the relative significance of different units cannot be reliably estimated from mutual dependencies between units as suggested in related research. Therefore, data units can be classified reliably into different perceptual priority classes via analysis by synthesis only. For estimating the co-impact of multiple losses, we propose a simple loss-distortion model. The model can be used for optimizing UEP in practical streaming systems.
Fine-granular scalable and error resilient audio coding by tree-structured quantization

This paper presents a scheme for scalable, error resilient coding of audio signals. The scheme is implemented here as part of a subband-based audio codec and involves the union of two interconnected ideas: tree-structured vector quantization (TSVQ) and reallocation of bits (RoB). The former ensures that the encoding procedure results in fixed-length codes while the latter is a low-complexity scaling algorithm that computes optimal bitrates for each subband when the bitstream needs to be downscaled to a lower bitrate. The functionality enabled by these two systems assumes great importance when the encoded material is to be delivered over channels and networks that have a time-varying bitrate, such as IP-based wireless networks. Such channels are often error-prone and the potentially debilitating effects of introduced errors are mitigated by the inherent fixed-length property of all codewords. We present results that highlight the performance of this system under noisy conditions and show that the degradation in quality is graceful.
Design and storytelling concepts in the quality of experience

In this paper, we examine the implicit aspects of quality activated by the design and storytelling components of the consumer's product and the consumption environment. We describe what affects the consumers' assessment of the product by looking beyond its surface. We review several design and marketing examples including previous research in multimedia and information processing and other fields to understand the how the unconscious realm of thoughts and impressions can create an impact on quality judgement. We also propose a framework in the Quality of Experience that includes the implicit aspects of quality judgement.
Efficient Smoothing of Robust VBR Video Traffic by Explicit Slice-based Mode Type Selection

Variable bit rate encoding is considered to outper- form constant bit rate encoding with regard to compression gain and constancy of the video quality. The drawback is, however, that variable bit rate video streams usually exhibit a significant degree of burstiness which may result in poor multiplexing properties, buffer overflows, and large network delays. In this work we show how the explicit selection of slice mode types can effectively be used to generally avoid the use of large intracoded frames while still providing robustness against packet losses in the network. The employed encoding scheme provides a considerable smoothing of variable bit rate video traffic, resulting in less delay, less delay jitter, and less loss due to buffer overflows and late arrivals. It preserves the efficiency of variable bit rate video encoding, has little overhead, and is in accordance with current video coding standards.