nils-herrmann/clean_scirepeval huggingface dataset:
Analysis of SSIM performance for Digital Cinema applications

Visual quality is one of the most important issues in Digital Cinema applications, and the most practical method to measure visual quality is using objective metrics. Several objective metrics that take into account perceived visual quality have been developed in the past few years; one of them is based on the structural similarity paradigm. To analyze the performance of these objective metrics for digital cinema applications, a subjective visual quality assessment in digital cinema environments using contents from the Digital Cinema Initiative (DCI) Standard Evaluation Material has been conducted in a previous study. Since in practice digital cinema utilizes only high quality imagery, in this paper we analyze further a subset of highquality stimuli from the previous study.
A temporal error concealment algorithm based on structural alignment for packet video

With rapid growth of packet switched communication using the IP protocol, concealment of packet-loss related distortions in multimedia communications assumes great importance. This paper proposes a novel temporal concealment algorithm to conceal packet-loss related distortion in video transmission. The algorithm uses the previous frame (assumed to be error-free) to conceal the distortion in the present frame. It is based on the insight that the spatial region around the area in the previous frame that yields the best concealment will in general be highly correlated with the corresponding area around the region to be concealed in the present frame. We implement the proposed algorithm for a MPEG-2 based packet-loss scenario and compare its performance with prevalent algorithms based on boundary matching. The proposed algorithm effects better concealment as compared to these algorithms. In particular, it exhibits a superior ability to align edges. With its low computation complexity, this algorithm can find application in real-time concealment of packet-loss related artifacts in streaming video.
Exploring diverse measures for evaluating QoE in the context of WebRTC

Enabling high Quality of Experience (QoE) with video-conferencing solutions based on Web Real-Time Communication (WebRTC) protocols anywhere, anytime, is challenging and triggers the exploration of new ways to gain QoE insights. In this paper, we share initial observations from a within-subjects experiment (N = 22) in which 2-party WebRTC-based audiovisual conversations took place under varying technical conditions. We collected self-report data, peripheral physiological data and application-level performance statistics with the overall aim of exploring their usefulness and compatibility for assessing QoE in the context of WebRTC. Our preliminary findings indicate that the varying quality is well-reflected in the self-reported overall quality and annoyance. However, this might not apply when considering the emotional valence ratings and ECG data, implying that other factors may play a role here.
On the Properties of Mean Opinion Scores for Quality of Experience Management

For research on quality of experience (QoE), mean opinion scores (MOS) are widely chosen as the results of subjective tests and the ground-truth reference for further research on objective quality modeling. Furthermore, the results of objective quality modeling are used for QoE management subsequently. Therefore, the performance of QoE management process actually depends heavily on MOS. However, the rationality of MOS for QoE management is not yet technically proven in the literature. In this paper, we first prove that subject homogeneity is implicitly assumed for obtaining MOS by modeling the arithmetic averaging process from a systematic viewpoint. However, we point out that actually subjects exhibit variability in terms of quality assessment. Then we elaborate that this mismatch may results in failures if we conduct QoE management based on MOS. Finally we propose a utility-based averaging method (uMOS) which improves the performance of QoE management.
A perceptual quality metric for stereoscopic crosstalk perception

Compared to metrics proposed to assess the quality of two-dimensional (2D) images, there are very few metrics devoted to quality assessment of stereoscopic presentations. Crosstalk is one of the most annoying distortions in the visualization stage of stereoscopic imaging technology. This paper proposes a perceptual quality metric which takes characteristics of stereoscopic images into account for predicting quality levels of crosstalk perception in stereoscopic images, based on an understanding of three main factors, crosstalk level, camera baseline and scene content. The experimental results demonstrate that the proposed metric has Pearson correlation of 87.7% when compared to the ground truth results from the subjective experiments on the crosstalk perception, which is much better than the traditional 2D metrics without integrating 3D depth information.
On the Error Resilience of Rate Smoothing using Explicit Slice-Based Mode Selection

An encoder based rate smoothing scheme which uses explicit slice-based mode selection has been proposed. The algorithm provides significantly smoother bitstream from a variable bit rate (VBR) encoder and reduces the network queuing delay while maintaining quality for the end-user. In this paper we investigate the error robustness of the proposed scheme based on H.264/AVC codec. The results show that, the proposed scheme maintains error resilience properties without extra delay. Compared to the standard frame based scheme, the algorithm provides almost the same average distortion while reducing the distortion variance which helps improving the subjective video quality. We investigate the performance in random and bursty packet loss environments, and it shows that the proposed scheme is applicable for error prone networks, for example, wireless networks.
Comparison of JPEG 2000 and H.264/AVC by subjective assessment in the Digital Cinema

Two video coding schemes with variable bit rates — JPEG 2000 and H.264/AVC — were compared in terms of perceived quality performance in a Digital Cinema environment. In this paper we describe in detail the procedure for a subjective quality assessment. The stimuli used were 10 seconds long color sequences at HD resolution (1920×1080 progressive), 30 fps, and YCbCr 4∶2∶0. We do not consider DCI-specified content, but rather exploring the quality of rates suitable for alternative content. The results show that temporal compression schemes like H.264/AVC can play out their high coding efficiency not only at SD resolutions, but also at high resolutions and at high bit rates around 31 Mbps.
Comparison of unequal erasure protection schemes for video streaming

Several different schemes for unequal error/erasure protection (UEP) in video streaming have been proposed during the past years. However, it is not a trivial task to define the optimal relative protection levels for different media units (ie. the basic units of decoding handled individually by media decoder) in practical streaming applications. In this paper, we use a theoretical loss-distortion model to evaluate the performance of different UEP schemes in realistic video streaming scenarios with fixed redundancy overhead budget. Our results confirm the observation that equal error/erasure protection (EEP) is desired when the packet loss rate is low compared to the redundancy overhead. The higher the packet loss rate, the more segregating UEP should be used to achieve optimal performance. The results also suggest that comparable performances can be obtained by using two protection levels only (protected part and unprotected part) instead of more complex UEP schemes.
Attention modeling for video quality assessment: Balancing global quality and local quality

This paper proposes to evaluate video quality by balancing two quality components: global quality and local quality. The global quality is a result from subjects allocating their attention equally to all regions in a frame and all frames in a video. It is evaluated by image quality metrics (IQM) with averaged spatiotemporal pooling. The local quality is derived from visual attention modeling and quality variations over frames. Saliency, motion, and contrast information are taken into account in modeling visual attention, which is then integrated into IQMs to calculate the local quality of a video frame. The local quality of a video sequence is calculated by pooling local quality values over all frames with a temporal pooling scheme derived from the known relationship between perceived video quality and the frequency of temporal quality variations. The overall quality of a distorted video is a weighted average between the global quality and the local quality. Experimental results demonstrate that the combination of the global quality and local quality outperforms both sole global quality and local quality, as well as other quality models, in video quality assessment. In addition, the proposed video quality modeling algorithm can improve the performance of image quality metrics on video quality assessment compared to the normal averaged spatiotemporal pooling scheme.
Multi-service load balancing in a heterogeneous network

In this paper, we investigate load balancing mechanisms in heterogeneous WiMAX/WLAN network. By taking both the service characteristics of the two networks and service requirements of two kinds of applications into consideration, we distribute all streaming applications to WiMAX with preemptive service priority. Then the remaining capacities of WiMAX and the entire capacity of WLAN are utilized for serving elastic applications. Accordingly, we proposed two size thresholds based on which elastic applications are dispatched to WiMAX or WLAN. Numerical results indicate that dispatching elastic applications with smaller size to WLAN performs fair well compared with other dispatching mechanisms. It is also found that the size distribution has significant impact on the performance of load balancing mechanisms.
Attention Driven Foveated Video Quality Assessment

Contrast sensitivity of the human visual system to visual stimuli can be significantly affected by several mechanisms, e.g., vision foveation and attention. Existing studies on foveation based video quality assessment only take into account static foveation mechanism. This paper first proposes an advanced foveal imaging model to generate the perceived representation of video by integrating visual attention into the foveation mechanism. For accurately simulating the dynamic foveation mechanism, a novel approach to predict video fixations is proposed by mimicking the essential functionality of eye movement. Consequently, an advanced contrast sensitivity function, derived from the attention driven foveation mechanism, is modeled and then integrated into a wavelet-based distortion visibility measure to build a full reference attention driven foveated video quality (AFViQ) metric. AFViQ exploits adequately perceptual visual mechanisms in video quality assessment. Extensive evaluation results with respect to several publicly available eye-tracking and video quality databases demonstrate promising performance of the proposed video attention model, fixation prediction approach, and quality metric.
A good quality, low complexity 4.8 kbit/s stochastic multipulse coder

A fully quantized, 4.8-kb/s, code-excited, linear predictive (CELP) coder using a stochastic-multipulse codebook and a self-excitation sequence (SESTMP) is presented. The coder is compared to a standard CELP coder with respect to both complexity and quality. It is shown that the SESTMP is far less complex with little perceivable degradation in quality. Implementation on commercially available digital signal processors is discussed.<<ETX>>
An MPEG-21 framework for streaming media

The MPEG-21 vision is to define a multimedia framework to enable transparent and augmented use of multimedia resources across a wide range of networks and devices used by the different communities. This paper describes the evolution of a test bed simulating an MPEG-21 framework considering a streaming media use case scenario. The test bed has been set up in order to realize, validate and perform experimentations on streaming media considering the integration of MPEG-4 content representation and MPEG-7 audiovisual content description interface within the MPEG-21 multimedia framework. The application scenario is within the concept of universal multimedia access (UMA) realizing the challenge of "create once play any time, any how, any where".
An efficient packetization algorithm for JPEG2000

The paper presents an efficient algorithm for post compression optimal rate allocation and packetization within JPEG2000 encoding. JPEG2000, the new ISO/ITU-T standard for still image coding, has been shown to provide superior coding efficiency to the previous standard, JPEG. However, the added efficiency of JPEG2000 comes at the cost of increased computational requirements. To improve the computational efficiency of JPEG2000, we propose a new algorithm for JPEG2000 rate allocation and packetization utilizing the D-heap data structure. Implemented in Jasper and tested on five reference images, this algorithm provides a speedup for JPEG2000's rate allocation and packetization of 15.9 times on average, and enables an average overall speedup of 33% for JPEG2000 encoding.
Multi-level cache hierarchy evaluation for programmable media processors

This paper presents the results of a multi-level cache memory hierarchy evaluation for programmable media processors. With the continuing advances in VLSI technology, it becomes possible to support larger memory hierarchies on-chip, but the question remains of how to most effectively use these additional silicon resources for optimizing memory performance. This paper explores that issue by evaluating the various levels of the memory hierarchy using a cache-based memory system. This evaluation examines the change in performance from varying cache parameters including the L2 cache parameters of cache size, line size, and latency, and the external memory parameters of bandwidth and latency. Examining the performance impact of these parameters, we have identified external memory latency and bandwidth as the primary memory bottlenecks in media processors.
Parallel media processors for the billion-transistor era

This paper describes the challenges presented by single-chip parallel media processors (PMPs). These machines integrate multiple parallel function units, instruction execution, and memory hierarchies on a single chip. The combination of programmability and high performance on data parallelism is necessary to meet the demands of next-generation multimedia applications. Many research issues must be solved to realize the full potential of programmable media processors. This paper provides both a survey of research trends and issues in architecture and compiler design for programmable media processors, and an exploration of the potential performance of media processors over the next decade.
Datapath design for a VLIW video signal processor

This paper represents a design study of the datapath for a very long instruction word (VLIW) video signal processor (VSP). VLIW architectures provide high parallelism and excellent high-level language programmability, but require careful attention to VLSI and compiler design. Flexible, high-bandwidth interconnect, high-connectivity register files, and fast cycle times are required to achieve real-time video signal processing. Parameterizable versions of key modules have been designed in a 0.25 /spl mu/m process, allowing us to explore tradeoffs in the VLIW VSP design space. The designs target 33 operations per cycle at clock rates exceeding 600 MHz. Various VLIW code scheduling techniques have been applied to 6 VSP kernels and evaluated on 7 different candidate datapath designs. The results of these simulations are used to indicate which architectural tradeoffs enhance overall performance in this application domain.
Dynamic parallel media processing using speculative broadcast loop (SBL)

This paper presents the results of a study of dynamic parallel media processing using Speculative Broadcast Loop (SBL), a speculative run-time looplevel parallelization method. Due to processing regularity, multimedia applications typically contain extensive parallelism. Subword parallelism methods are commonly used to support data parallelism between independent loop iterations in inner loops, but much of the data parallelism in media processing resides in outer loops and cannot be supported with subword parallelism. Larger-scale parallel methods are needed to enable use of the full range of data parallelism in multimedia. Because static parallel compilation methods are often unable to recognize all parallelism at compile time, a run-time method is assumed for the speculative execution of potentially parallel loops. The SBL run-time method combines SIMD parallelism with large-scale speculation for supporting data parallelism in multimedia.
Real-time parallel MPEG-2 decoding in software

The growing demand for high quality compressed video has led to an increasing need for real-time MPEG decoding at greater resolutions and picture sizes. With the widespread availability of small-scale multiprocessors, a parallel software implementation may provide an effective solution to the decoding problem. We present a parallel decoder for the MPEG standard, implemented on a shared memory multiprocessor. Goal of this work is to provide an all-software solution for real-time, high-quality video decoding and to investigate the important properties of this application as they pertain to multiprocessor systems. Both coarse and fine grained implementations are considered for parallelizing the decoder. The coarse-grained approach exploits parallelism at the group of pictures level, while the fine-grained approach parallelizes within pictures, at the slice level. A comparative evaluation of these methods is made, with results presented in terms of speedup, memory requirements, load balance, synchronization time, and temporal and spatial locality. Both methods demonstrate very good speedups and locality properties.
Production-Quality Video Over Broadband Networks: A System Description and Two Interactive Applications

The Washington University MultiMedia eXplorer (MMX) is a complete, host-independent multimedia system capable of transmitting and receiving JPEG-compressed video, CD-quality audio, and high-resolution radiographic images over the Washington University broadband ATM network. If the host is equipped with an ATM interface card, normal network traffic can be supported via an ATM extension port on the MMX. The major components of the MMX are an ATMizer and three multimedia channels. The ATMizer implements the host interface, the interface to the ATM network, and the interface to the three multimedia channels. This paper describes the architecture of the MMX, the software used with the system, and two applications that have been developed to demonstrate the capability of broadband ATM networks for multimedia applications.
Localized Content-Based Image Retrieval

We define localized content-based image retrieval as a CBIR task where the user is only interested in a portion of the image, and the rest of the image is irrelevant. In this paper we present a localized CBIR system, Accio, that uses labeled images in conjunction with a multiple-instance learning algorithm to first identify the desired object and weight the features accordingly, and then to rank images in the database using a similarity measure that is based upon only the relevant portions of the image. A challenge for localized CBIR is how to represent the image to capture the content. We present and compare two novel image representations, which extend traditional segmentation-based and salient point-based techniques respectively, to capture content in a localized CBIR setting.
Meta-Evaluation of Image Segmentation Using Machine Learning

Image segmentation is a fundamental step in many computer vision applications. Generally, the choice of a segmentation algorithm, or parameterization of a given algorithm, is selected at the application level and fixed for all images within that application. Our goal is to create a stand-alone method to evaluate segmentation quality. Stand-alone methods have the advantage that they do not require a manually-segmented reference image for comparison, and can therefore be used for real-time evaluation. Current stand-alone evaluation methods often work well for some types of images, but poorly for others. We propose a meta-evaluation method in which any set of base evaluation methods are combined by a machine learning algorithm that coalesces their evaluations based on a learned weighting function, which depends upon the image to be segmented. The training data used by the machine learning algorithm can be labeled by a human, based on similarity to a human-generated reference segmentation, or based upon system-level performance. Experimental results demonstrate that our method performs better than the existing stand-alone segmentation evaluation methods.
Multi-level memory prefetching for media and stream processing

This paper presents a multi-level memory prefetch hierarchy for media and stream processing applications. Two major bottlenecks in the performance of multimedia and network applications are long memory latencies and limited off-chip processor bandwidth. Aggressive prefetching can be used to mitigate the memory latency problem, but overly aggressive prefetching may overload the limited external processor bandwidth. To accommodate both problems, we propose multilevel memory prefetching. The multi-level organization enables conservative prefetching on-chip and more aggressive prefetching off-chip. The combination provides aggressive prefetching while minimally impacting off-chip bandwidth, enabling more efficient memory performance for media and stream processing. This paper presents preliminary results for multi-level memory prefetching, which show that combining prefetching at the L1 and DRAM memory levels provides the most effective prefetching with minimal extra bandwidth.
Efficient rate control for JPEG-2000

In this paper, we present two new methods for efficient rate control and entropy coding in lossy image compression using JPEG-2000. These two methods enable significant improvements in computation complexity and power consumption over the traditional JPEG-2000 algorithms. First, we propose a greedy heap-based rate-control algorithm (GHRaC), which achieves efficient postcompression rate control by implementing a greedy marginal analysis method using the heap sort algorithm. Second, we propose an integrated rate-control and entropy-coding (IREC) algorithm that reduces the computation complexity of entropy coding by selectively entropy coding only the image data that is likely to be included in the final bitstream, as opposed to entropy coding all image data. Together, these two methods enable significant savings in computation time and power consumption. For example, the GHRaC method demonstrates 16 /spl times/ speedup for rate control when encoding the Lena color image using a target compression ratio of 128:1, one quality layer, and code blocks of 32 /spl times/ 32 pixels. The IREC method expands upon GHRaC to perform entropy coding in conjunction with rate control. Using an enhanced version of IREC, these two methods jointly achieve a speedup in execution time of 14 /spl times/ over traditional rate control and entropy coding, which first entropy codes all image coefficients and then separately performs postcompression rate control using the generalized Lagrange multiplier method to select which data are included in the final bitstream. Both theoretical analysis and empirical results are presented in validating the advantages of the proposed methods.