nils-herrmann/clean_scirepeval huggingface dataset:
Comparison of subjective viewing test methods for image quality assessment

This paper presents a comparison study on subjective quality scores obtained by both single stimulus (without reference) and triple stimulus (with reference) methods. The TID2013 database is reevaluated by single stimulus approach, which is realized by absolute category rating (ACR). And the mean opinion score (MOS) provided along with TID2013 represents the results from triple-stimulus pair comparison (3-stimulus PC) method. In the end, the correlation coefficient and hypothesis testing are used to determine if there is a significant difference between both sets of scores. The experimental results show that the differences exist and are significant for some specific distortion types or image contents. We believe the findings in this work can benefit and facilitate the future subjective visual quality viewing tests.
A Flexible Starting Point Based Partial Caching Algorithm for Video on Demand

In this paper, we propose a novel proxy caching scheme for Video on Demand (VoD) services. Our approach is based on an observation we have made during subjective VoD performance evaluation tests. We have found that when users are seeking for some specific content, they pay most attention to the initial delay, while a small shift of the starting point is acceptable. Based on this observation as well as the dynamic popularity of video segments, we propose an efficient segment based caching algorithm. The approach is applied on a proxy and minimizes the average initial playout delay at the clients. Our experimental results show a significant reduction of the average initial waiting time compared to conventional caching approaches.
A Probabilistic Appearance Representation and Its Application to Surprise Detection in Cognitive Robots

In this work, we present a novel probabilistic appearance representation and describe its application to surprise detection in the context of cognitive mobile robots. The luminance and chrominance of the environment are modeled by Gaussian distributions which are determined from the robot's observations using Bayesian inference. The parameters of the prior distributions over the mean and the precision of the Gaussian models are stored at a dense series of viewpoints along the robot's trajectory. Our probabilistic representation provides us with the expected appearance of the environment and enables the robot to reason about the uncertainty of the perceived luminance and chrominance. Hence, our representation provides a framework for the detection of surprising events, which facilitates attentional selection. In our experiments, we compare the proposed approach with surprise detection based on image differencing. We show that our surprise measure is a superior detector for novelty estimation compared to the measure provided by image differencing.
On the role of multimodal communication in telesurgery systems

Telesurgery systems integrate multimodal communication and robotic technologies to enable surgical procedures to be performed from remote locations. They allow human surgeons to intuitively control laparoscopic instruments and to navigate within the human body. In this paper, we present selected topics on multimodal interaction in the context of telesurgery applications. These are results from the collaborative research project SFB 453 on “High-Fidelity Telepresence and Teleaction” which is funded by the German Research Foundation in the larger Munich area. The focus in this paper is on multimodal information processing and communication including simulation of surgical targets in the human body. Furthermore, we present an overview of our advanced multimodal telesurgery demonstrators that provide a comprehensive platform for our collaborative telepresence research.
Towards an objective quality evaluation framework for haptic data reduction

High packet rates in telepresence and teleaction systems pose grave challenges to teleoperation over existing communication infrastructure like the Internet. To counter these issues, efficient perceptually motivated packet-rate reduction schemes have been developed. These schemes are conventionally evaluated for perceived quality via subjective user tests. Such tests are time-consuming, expensive and require precise control of experimental conditions. Computer modeling of telepresence sessions can, on the other hand, bring repeatability, ease of observation, definite control over system parameters and task description and fairness of comparison. In this paper, we present first steps towards a methodology and a framework to model and simulate a networked haptic interaction and evaluate it objectively for the quality of experience. Towards this purpose, we model the human control action and haptic perception process in teleoperation. Our results show that simulations of these models for a range of data reduction scheme parameters produce quality estimates whose trend is comparable to carefully performed subjective user tests.
Real-time compression of point cloud streams

We present a novel lossy compression approach for point cloud streams which exploits spatial and temporal redundancy within the point data. Our proposed compression framework can handle general point cloud streams of arbitrary and varying size, point order and point density. Furthermore, it allows for controlling coding complexity and coding precision. To compress the point clouds, we perform a spatial decomposition based on octree data structures. Additionally, we present a technique for comparing the octree data structures of consecutive point clouds. By encoding their structural differences, we can successively extend the point clouds at the decoder. In this way, we are able to detect and remove temporal redundancy from the point cloud data stream. Our experimental results show a strong compression performance of a ratio of 14 at 1 mm coordinate precision and up to 40 at a coordinate precision of 9 mm.
An area-of-interest based communication architecture for Shared Haptic Virtual Environments

Communication architectures conceived for Shared Haptic Virtual Environments (SHVEs) are based on either the client-server or the peer-to-peer paradigms. High-rate rendering and communication between collaborating users quickly leads to performance bottlenecks, if the virtual environment's size and complexity or the number of collaborating users increases. We propose a decentralized communication architecture for SHVEs, which exploits areas of interest (AoI) of a user to dynamically form smaller communication groups. High-rate information exchange following the client-server paradigm is used within these groups to support satisfactory haptic collaboration and consistency. A single group member is selected to simulate the object states and to relay this state to the other group members. Peer-to-peer inter-group communication is also used, based on spatial proximity, to further reduce the overall communication load. We implemented a prototype based on our proposed architecture and present some first evaluation results. They serve as a proof-of-concept and show the effectiveness of dynamic group maintenance in conjunction with additional traffic control schemes.
A novel full-reference video quality metric and its application to wireless video transmission

In this paper, we present a novel objective video quality metric that captures the trade-off between the picture quality and the temporal resolution of a compressed video. The proposed metric is based on PSNR, frame rate as well as spatial and temporal activity measures that are obtained from the video. The content-independency of the metric makes it useful for the dynamic optimization of wireless video transmission. With the proposed metric, it is possible to adjust the trade-off between spatial and temporal qualities such that the user satisfaction is maximized. Our metric is very accurate, as verified by statistical analysis with data from subjective tests. We integrate the metric into a real-time wireless video transmission system and show in our experiments that the system, with our metric's ability to predict perceptual quality, can deliver significantly improved perceptual quality for arbitrary videos over a wide range of channel conditions.
Multi-source video multicast in peer-to-peer networks

We propose a novel framework for multi-source video streaming in peer-to-peer (P2P) networks. Multiple videos are distributed to all requesting peers exploiting full collaboration between the sources, the requesting peers and helper peers. Each source distributes its own video sequence while additionally forwarding video data received from other sources. A single peer is selected to redistribute a particular video block to the peers which would like to receive the videos. Our goal is to maximize the overall throughput or alternatively the aggregate video quality of multiple concurrent streaming sessions. We also consider the special cases of "same throughput" or "same video quality" for all streams. We formulate the rate allocation and redistribution as an optimization problem and evaluate our framework for three different scenarios. In the first scenario, the rate allocation is jointly decided for all participating peers. In the second scenario, the rate allocation is also decided jointly, but additionally either same rate or same video quality streams are enforced. Our third scenario assumes separate distribution for every source. In this case, the peers divide their upload capacity equally among the different video sequences. Our results show the superior performance of joint rate allocation compared to independent allocation and the effectiveness of our framework.
Flexible distribution of computational complexity between the encoder and the decoder in distributed video coding

Based on our previous work on distributed video coding with low complexity motion estimation at the encoder, we propose in this paper a more flexible system, providing the possibility that the encoder and the decoder cooperate in motion estimation and share the computational bulk of it. In this system, not only the encoder but also the decoder is capable of improving the coding efficiency by spending more computational power in motion estimation. Experimental results show that the cooperation of the encoder and the decoder can reduce the total computational complexity and at the same time improve the coding efficiency. In addition, we propose a simple scheme to order the intensity of motion activity for different macroblocks. When the encoder has limited computational power, we apply motion estimation only to the macroblocks with relatively large motion. This leads to an efficient use of computational power at the encoder.
Wyner-Ziv Video Coding Based on Turbo Codes Exploiting Perfect Knowledge of Parity Bits

In a Wyner-Ziv video coding system based on turbo codes, parity bits are generated for a Wyner-Ziv frame and transmitted to the decoder. In this context of distributed source coding, we can assume that the transmission of the parity bits is guaranteed to be error-free by the lower network layers. In our work, we propose a simple method to exploit the guaranteed reliability of the parity bits so that the turbo decoding achieves a better performance. The proposed scheme is to assign the parity bits a higher weight than the information bits in turbo decoding. Our simulation on binary sequences shows that the proposed scheme enables turbo codes to come closer to the Slepian-Wolf bound in distributed lossless source coding. When it is applied to Wyner-Ziv video coding, the bit rate is reduced by 2% -4% for a given video quality. Although the gain is relatively small, it comes for free, because all that we have to do is to multiply the parity bits with a sufficiently large factor before the decoding process starts.
Evaluation of segment-based proxy caching for video on demand

In this paper, we derive an analytical model for the evaluation of the performance of a video on demand (VoD) system. The model estimates the mean waiting time achievable by the popularity-aware partial caching (PAPA) algorithm from our previous work. Two approximation strategies are proposed for low computational complexity. Furthermore, we also consider the influence of a starting point shift on the quality of experience and combine the two factors into a universal user satisfaction metric. In order to find the relation between the two impairments, waiting time and starting point shift, sophisticated subjective tests are performed. With the final score model, a more comprehensive evaluation of the system can be obtained with very low computational complexity.
Room segmentation in 3D point clouds using anisotropic potential fields

Emerging applications, such as indoor navigation or facility management, present new requirements of automatic and robust partitioning of indoor 3D point clouds into rooms. Previous research is either based on the Manhattan-world assumption or relies on the availability of the scanner pose information. We address these limitations by following the architectural definition of a room, where the room is an inner free space separated from other spaces through openings or partitions. For this we formulate an anisotropic potential field for 3D environments and illustrate how it can be used for room segmentation in the proposed segmentation pipeline. The experimental results confirm that our method outperforms state-of-the-art methods on a number of datasets including those that violate the Manhattan-world assumption.
Spatial resolution of vibrotactile perception on the human forearm when exploiting funneling illusion

Recent advances in man-machine interaction, telerobotics, telepresence and teleaction have shown that introducing the haptic modality to multimedia applications has the power to significantly widen their application range and to dramatically improve the user experience. Especially, the human body surface has been considered as an additional means of presenting information using vibrotactile display devices. In this context, spatial displacement of a vibrotactile stimulus can be deployed for information display. By exploiting a psychophysical illusion called “funneling illusion”, we are able to increase the spatial resolution of vibrotactile displays. In this paper, we aim at investigating the spatial resolution of vibrotactile perception on the human forearm when applying multiple “funneling” stimuli. In our psychophysical experiments, we revealed the human spatial perception ability on the human forearm for stationary and moving vibrotactile stimuli.
Trends in Multimedia Signal Processing [In the Spotlight]

Pushed by the tremendous technological and societal changes of the last decade, multimedia is emerging more and more as the cornerstone of next generation information and communication technologies. Mobile devices, sensors, embedded systems, high-performance computing, broadband networks, and 3-D are some of the technologies that are pervading a generation of users that are more and more exigent and proactive. Evolving from a simple integration of existing technologies to a mutually aware, interdisciplinary development of new technologies is the biggest challenge of the multimedia SP (MMSP) community. In this article, we will list some of the most promising research trends that have recently emerged in MMSP.
Dynamic model displacement for model-mediated teleoperation

In this paper, we study and extend the concept of model-mediated teleoperation (MMT) for teleaction systems which provide live video feedback from the remote side with a time delay. In MMT, the haptic feedback is rendered locally on the operator side using a simple object surface model in order to keep the haptic control loop stable in the presence of communication delays. Because the live video from the remote side is received with delay, this results in a visual-haptic asynchrony for the displayed interaction events. In addition, sudden model parameter updates can lead to “model-jump” effects for the displayed haptic feedback. Both effects degrade the user experience and system performance. To address these issues, we propose an extension of MMT which we call model-displaced teleoperation (MDT) in this paper. In MDT, we adaptively shift the position of the local surface model to delay the haptic contact with the environment, thus compensating the visual-haptic asynchrony and avoiding the model-jump effect. As the haptic feedback is still rendered locally, the advantages of the MMT approach are retained and instabilities in the haptic interaction are avoided. In our experiments, we determine the optimal displacement compromise between visual-haptic asynchrony, the model-jump effect and perceived distance errors. Moreover, the subjective experience and objective task performance of the proposed MDT and the original MMT for a teleoperation setup with soft objects are evaluated. Our results show that the users prefer the MDT method compared to MMT once the communication delay between the teleoperator and the operator exceeds 50ms. In addition, the task error rate is reduced by about 50% and the subjects are better able to control their contact force for system delays larger than 50ms if the MDT method is employed.
A system for high precision glass-to-glass delay measurements in video communication

Ultra low delay video transmission is becoming increasingly important. Video-based applications with ultra low delay requirements range from teleoperation scenarios such as controlling drones or telesurgery to autonomous control of dynamic processes using computer vision algorithms applied on real-time video. To evaluate the performance of the video transmission chain in such systems, it is important to be able to precisely measure the glass-to-glass (G2G) delay of the transmitted video. In this paper, we present a low-complexity system that takes a series of pairwise independent measurements of G2G delay and derives performance metrics such as mean delay or minimum delay etc. from the data. The precision is in the sub-millisecond range, mainly limited by the sampling rate of the measurement system. In our implementation, we achieve a G2G measurement precision of 0.5 milliseconds with a sampling rate of 2kHz.
Analysis of distortion due to packet loss in streaming video transmission over wireless communication links

In this paper, we provide an accurate and fully analytical model for the distortion due to lost frames in wireless video transmission. Our analysis combines the properties of the video sequence and those of the wireless transmission link. Built on the long-term average properties of the video source, we first develop a model for the average distortion due to the loss of frames, and then derive a mathematical model for analysis of transmission errors occurring during wireless transmission. Experimental results show a surprisingly accurate behaviour of the proposed model.
QoE-Based Cross-Layer Optimization of Wireless Video with Unperceivable Temporal Video Quality Fluctuation

This paper proposes a novel approach for Quality of Experience (QoE) driven cross-layer optimization for wireless video transmission. We formulate the cross-layer optimization problem with a constraint on the temporal fluctuation of the video quality. Our objective is to minimize the temporal change of the video quality as perceivable quality fluctuations negatively affect the overall quality of experience. The proposed QoE scheme jointly optimizes the application layer and the lower layers of a wireless protocol stack. It allocates network resources and performs rate adaptation such that the fluctuations lie within the range of unperceivable changes. We determine corresponding perception thresholds via extensive subjective tests and evaluate the proposed scheme using an OPNET High Speed Downlink Packet Access (HSDPA) emulator. Our simulation results show that the proposed approach leads to a noticeable improvement of overall user satisfaction for the provided video delivery service when compared to state-of-the-art approaches.
Automatic reconstruction of stationary 3-D objects from multiple uncalibrated camera views

A system for the automatic reconstruction of real-world objects from multiple uncalibrated camera views is presented. The camera position and orientation for all views, the 3-D shape of the rigid object, as well as the associated color information, are recovered from the image sequence. The system proceeds in four steps. First, the internal camera parameters describing the imaging geometry are calibrated using a reference object. Second, an initial 3-D description of the object is computed from two views. This model information is then used in a third step to estimate the camera positions for all available views using a novel linear 3-D motion and shape estimation algorithm. The main feature of this third step is the simultaneous estimation of 3-D camera-motion parameters and object shape refinement with respect to the initial 3-D model. The initial 3-D shape model exhibits only a few degrees of freedom and the object shape refinement is defined as flexible deformation of the initial shape model. Our formulation of the shape deformation allows the object texture to slide on the surface, which differs from traditional flexible body modeling. This novel combined shape and motion estimation using sliding texture considerably improves the calibration data of the individual views in comparison to fixed-shape model based camera-motion estimation. Since the shape model used for model based camera-motion estimation is only approximate, a volumetric 3-D reconstruction process is initiated in the fourth step that combines the information from ail views simultaneously. The recovered object consists of a set of voxels with associated color information that describes even fine structures and details of the object. New views of the object can be rendered from the recovered 3-D model, which has potential applications in virtual reality or multimedia systems and the emerging field of video coding using 3-D scene models.
Low-Complexity No-Reference PSNR Estimation for H.264/AVC Encoded Video

We present a no-reference (NR) PSNR estimation method which is based on only two bitstream features (average bitrate and mean quantization parameter of the I-frames). The low computational complexity of the proposed method makes it suitable for in-network real-time applications. The NR metric achieves a Pearson correlation of 0.99 for individual videos and a RMSE of approximately 1 dB PSNR on average. We additionally investigate the effect of various encoding configurations on the PSNR and show the robustness of our method towards these. Finally, we incorporate the proposed metric into an example application and demonstrate that only a minor performance loss is observed compared to the reference scheme which assumes the availability of true PSNR information.
Joint calibration of a camera triplet and a laser rangefinder

This paper studies the calibration of a multi-sensor capture device for the acquisition of image-based scene representations. It is composed of cameras and a laser rangefinder. We use a plane based approach to calibrate the compound system, determining both intrinsic and extrinsic parameters. The novelty of our approach is that the cameras and the scanner are treated jointly. We have tested our method both in simulations and on real data and observe very good calibration results.