nils-herrmann/clean_scirepeval huggingface dataset:
Binaural Source Localization and Spatial Audio Reproduction for Telepresence Applications

Telepresence is generally described as the feeling of being immersed in a remote environment, be it virtual or real. A multimodal telepresence environment, equipped with modalities such as vision, audition, and haptic, improves immersion and augments the overall perceptual presence. The present work focuses on acoustic telepresence at both the teleoperator and operator sites. On the teleoperator side, we build a novel binaural sound source localizer using generic Head Related Transfer Functions (HRTFs). This new localizer provides estimates for the direction of a single sound source given in terms of azimuth and elevation angles in free space by using only two microphones. It also uses an algorithm that is efficient compared to the currently known algorithms used in similar localization processes. On the operator side, the paper addresses the problem of spatially interpolating HRTFs for densely sampled high-fidelity 3D sound synthesis. In our telepresence application scenario the synthesized 3D sound is presented to the operator over headphones and shall achieve a high-fidelity acoustic immersion. Using measured HRTF data, we create interpolated HRTFs between the existing functions using a matrix-valued interpolation function. The comparison with existing interpolation methods reveals that our new method offers superior performance and is capable of achieving high-fidelity reconstructions of HRTFs.
Rule-Based No-Reference Video Quality Evaluation Using Additionally Coded Videos

This contribution presents a no-reference video quality metric, which is based on a set of simple rules that assigns a given video to one of four different content classes. The four content classes distinguish between video sequences which are coded with a very low data rate, which are sensitive to blocking effects, which are sensitive to blurring, and a general model for all other types of video sequences. The appropriate class for a given video sequence is selected based on the evaluation of feature values of an additional low-quality version of the given video, which is generated by encoding. The visual quality for a video sequence is estimated using a set of features, which includes measures for the blockiness, the blurriness, the spatial activity, and a set of additional continuity features. The way these features are combined to compute one overall quality value is determined by the feature class, to which the video has been assigned. We also propose an additional correction step for the visual quality value. The proposed metric is verified in a process, which includes visual quality values originating from subjective quality tests in combination with a cross validation approach. The presented metric significantly outperforms peak-signal-to-noise ratio as a visual quality estimator. The Pearson correlation between the estimated visual quality values and the subjective test results takes on values as high as 0.82.
Multi-modal multi-user telepresence and teleaction system

The video shows a rich multi-modal multi-user telepresence system, which was developed within the SFB453 funded by the German Research Foundation (www.sfb453.de). As a complex application scenario, the remote repairing of a broken pipe is presented in this paper. The system basically consists of two operator-teleoperator- pairs. While one of the operators interacts with a stationary human- system-interface, the other operator uses a mobile one. Both systems provide visual, auditory, and haptic feedback and enable to control the motion of head, arms, and grippers as well as the locomotion of the corresponding teleoperator.
A Trust Model for Intervehicular Communication Based on Belief Theory

Vehicles will exchange much information in the future in order to efficiently maintain their inner model of the environment. Before they can belief received pieces of information, they must evaluate their reliability. Trust is a mechanism to estimate this reliability based on the sender. As cars often drive the same route, they meet each other again and again. They can establish friendship-like relations and thus are embedded in a social structure. A trust model depends on this social structure. For this reason, we simulate the driving pattern of a small town. Within this simulation, all cars are equipped with a trust model that continuously monitors the experiences made with others. The developed model focuses on direct experiences of the individual and not on a system-wide reputation which would depend on a central unit. It continuously evaluates the performance and reputation of other cars and includes a feedback loop to faster adapt to changes in the other's behaviour. To make a decision out of the collected data, the model uses the capacity of the binary error and erasure channel from information theory. This capacity provides a better decision criterion than the traditional expectation value. The proposed trust model is an individual-level model; nonetheless it can be connected to a system-wide reputation mechanism.
On the use of reference monitors in subjective testing for HDTV

Most international standards recommend the use of reference monitors in subjective testing for visual quality. But do we really need to use reference monitors? In order to find an answer to this question, we conducted extensive subjective tests with reference, color calibrated high quality and uncalibrated standard monitors. We not only used different HDTV sequences, but also two fundamentally different encoders: AVC/H.264 and Dirac. Our results show that using the uncalibrated standard monitor, the test subjects underestimate the visual quality compared to the reference monitor. Between the reference and a less expensive color calibrated high quality monitor, however, we were unable to find a statistically significant difference in most cases. This might be an indication that both can be used equivalently in subjective testing, although further studies will be necessary in order to get a definitive answer.
Improving the visual quality of AVC/H.264 by combining it with content adaptive depth map compression

The future of video coding for 3DTV lies in the combination of depth maps and corresponding textures. Most current video coding standards, however, are only optimized for visual quality and are not able to efficiently compress depth maps. We present in this work a content adaptive depth map meshing with tritree and entropy encoding for 3D videos. We show that this approach outperforms the intra frame prediction of AVC/H.264 for the coding of depth maps of still images. We also demonstrate by combining AVC/H.264 with our algorithm that we are able to increase the visual quality of the encoded texture on average by 6 dB. This work is currently limited to still images but an extension to intra coding of 3D video is straightforward.
Improving the prediction accuracy of video quality metrics

To improve the prediction accuracy of visual quality metrics for video we propose two simple steps: temporal pooling in order to gain a set of parameters from one measured feature and a correction step using videos of known visual quality. We demonstrate this approach on the well known PSNR. Firstly, we achieve a more accurate quality prediction by replacing the mean luma PSNR by alternative PSNR-based parameters. Secondly, we exploit the almost linear relationship between the output of a quality metric and the subjectively perceived visual quality for individual video sequences. We do this by estimating the parameters of this linear relationship with the help of additionally generated videos of known visual quality. Moreover, we show that this is also true for very different coding technologies. Also we used cross validation to verify our results. Combining these two steps, we achieve for a set of four different high definition videos an increase of the Pearson correlation coefficient from 0.69 to 0.88 for PSNR, outperforming other, more sophisticated full-reference video quality metrics.
Path-finding using reinforcement learning and affective states

During decision making and acting in the environment humans appraise decisions and observations with feelings and emotions. In this paper we propose a framework to incorporate an emotional model into the decision making process of a machine learning agent. We use a hierarchical structure to combine reinforcement learning with a dimensional emotional model. The dimensional model calculates two dimensions representing the actual affective state of the autonomous agent. For the evaluation of this combination, we use a reinforcement learning experiment (called Dyna Maze) in which, the agent has to find an optimal path through a maze. Our first results show that the agent is able to appraise the situation in terms of emotions and react according to them.
Standards: The MPEG Open Access Application Format

The open access application format is an integrated exchange format that improves the management and exchange of content. It can enable Web sites, archives, or search engines to be able to perform automatic processing, indexing, and presentation of content. This method also allows the interoperable exchange of content between providers or between users. For the scientific community, this capability would enhance the visibility and comparability of the content and thus improve the organization and exchange of scientific literature. This would be helpful not only for the scientific community, but also other communities and organizations whose goal is to efficiently distribute and exchange open content.
Performance of Optical Flow Techniques on Graphics Hardware

Since graphics cards have become programmable the recent years, numerous computationally intensive algorithms have been implemented on the now called general purpose graphics processing units (GPGPUs). While the results show that GPGPUs regularly outperform CPU based implementations, the question arose how optical flow algorithms can be ported to graphics hardware. To answer the question, the optimal algorithm structure to maximize the performance gained by using graphics cards has to be found. In this paper we compare the performance of two algorithms that are highly different in structure, implemented on both CPU and graphics hardware. Analyzing the results of the CPU and GPGPU implementation, we explore the mapping of the algorithms to the graphics hardware and thereof extract information about a preferred structure of optical flow algorithms for GPGPU based implementation
Measuring Anthropometric Data for HRTF Personalization

Nowadays, multimodal human-like sensing, e.g. vision, hap tics and audition seeks to improve interaction between an operator (human) and a teleoperator (robot) in human centered robotic systems. Head Related Transfer Function (HRTF) based sound rendering techniques, which seek to create a realistic virtual auditory space for listeners, have become a prominent concept in human robot interaction. Applications that demand high quality 3D sound synthesis are usually based on measured HRTFs of listeners. Recently, researchers propose to construct a set of personalized HRTFs using multiple linear regression models between anthropometric data and measured HRTFs, which implies the existence of a training HRTF dataset together with the corresponding anthropometric data. This paper focuses on the measurement of Head-Related transfer Functions (HRTFs) and the corresponding anthropometric data of a listener. Several state-of-the-art techniques of measuring the HRTFs are described. For measuring the anthropometric data, we develop a low budget approach, which enables us to measure the anthropometry of a person within short time at a high accuracy, whereas the hardware costs for the scanning system are significantly reduced.
Complexity Constrained Robust Video Transmission for Hand-Held Devices

Robust video conversational applications for hand-held devices come with numerous challenges, e.g. real-time processing, complexity constrained devices and small end-to-end delays, etc. Transmission losses of compressed video data result in spatio-temporal error propagation in the decoded video sequence. To ensure some QoS, the video codec has to be well tuned to combat the degradation resulting from losses. Several feedback based error mitigation technique are assessed in this work. The proposed error robustness technique based on reference picture selection (RPS) and error tracking enhances the overall performance of the target system by more than 4 dB for moderate radio link control (RLC) PDU loss rates of 1.5%. This enhancement is achieved without any additional computational complexity.
Integrating a HRTF-based sound synthesis system into Mumble

This paper describes an integration of a Head Related Transfer Function (HRTF)-based 3D sound convolution engine into the open-source VoIP conferencing software Mumble. Our system allows to virtually place audio contributions of conference participants to different positions around a listener, which helps to overcome the problem of identifying active speakers in an audio conference. Furthermore, using HRTFs to generate 3D sound in virtual 3D space, the listener is able to make use of the cocktail party effect in order to differentiate between several simultaneously active speakers. As a result intelligibility of communication is increased.
Pose estimation of a moving humanoid using Gauss-Newton optimization on a manifold

Extracting the motion parameters of a moving humanoid is an important issue in machine vision. This is due to the need of emerging applications such as telepresence or robot navigation. In the context of telepresence, we consider a robot to be equipped with a stereo camera the pose of which has to be determined in a fast and robust way. The key issue is to compute the robust estimates of the (3times3) compatible essential matrices of the stereo cameras since the motion can then be uniquely estimated. In this paper, a robust technique that computes these entities is proposed under the assumption that the images are calibrated. The algorithm is based on the five-point relative pose problem using an optimization technique on a manifold. It determines these matrices by intersecting three essential manifolds. Test results show that the proposed method has a quadratic convergence rate and delivers accurate results. In addition, the algorithm has shown to exhibit better performance when compared to standard algorithms.
A Novel Humanoid Binaural 3D Sound Localization and Separation Algorithm

In this paper, we combine blind sound separation (BSS) and binaural localization for fast tracking of two concurrent sound sources. Using only two microphones placed inside the ear canals of a robot dummy head mounted on a torso and equipped with artificial ears, a generic set of head-related transfer functions (HRTFs) is used for accurate 3D source localization. This novel approach of combining the two BSS and system identification processes, and executing them simultaneously, is optimized for moving sound sources. Simulation results demonstrated precise localization of both elevation and azimuth angles of the two concurrent sound sources. The proposed method relies purely on auditive cues and uses a simple algorithm which, compared to the current localization algorithms, provides an easy implementation on robotic platforms and allows accurate as well as fast localization of concurrent sources
Robotic Localization and Separation of Concurrent Sound Sources using Self-Splitting Competitive Learning

We combine binaural sound-source localization and separation techniques for an effective deployment in humanoid-like robotic hearing systems. Relying on the concept of binaural hearing, where the human auditory 3D percepts are predominantly formed on the basis of the sound-pressure signals at the two eardrums, our robotic 3D localization system uses only two microphones placed inside the ear canals of a robot head equipped with artificial ears and mounted on a torso. The proposed localization algorithm exploits all the binaural cues encapsulated within the so-called head related transfer functions (HRTFs). Taking advantage of the sparse representations of the ear input signals, the 3D positions of two concurrent sound sources is extracted. The location of the sources is extracted after identifying which HRTFs they have been filtered with using a well-known self-splitting competitive learning clustering algorithm. Once the location of the sources are identified, they are separated using a generic HRTF dataset. Simulation results demonstrated highly accurate 3D localization of the two concurrent sound sources, and a very high signal-to-interference ratio (SIR) for the separated sound signals
A cognitive approach for a robotic welding system that can learn how to weld from acoustic data

Laser beam welding is the method of choice for the high-quality joining of materials. However, for industrial production these systems have to be set up and calibrated manually with much effort. Our objective is to apply intelligent data processing that results in a cognitive technical system that can learn how to weld, speed up the configuring process, and reduce costs. While monitoring laser welding with cameras and optical sensors has already been demonstrated elsewhere, this paper emphasizes the benefits of monitoring with acoustic sensors and feature extraction. Using acoustic sensors, the cognitive system is more sensitive to strong optical radiation. Several combined methods such as wavelet analysis, fast Fourier transformation, and linear dimensionality reduction are evaluated with sensor data from real experiments. Finally, as machine learning, the results are classified with learned reference data to obtain reliable information for monitoring and possibly using closed-loop control.
Depth map compression via compressed sensing

We propose in this paper a new scheme based on compressed sensing to compress a depth map. We first subsample the entity in the frequency domain to take advantage of its compressibility. We then derive a reconstruction scheme to recover the original map from the subsamples using a non-linear conjugate gradient minimization scheme. We preserve the discontinuities of the depth map at the edges and ensure its smoothness elsewhere by incorporating the Total Variation constraint in the minimization. The results we obtained on various test depth maps show that the proposed method leads to lower error rate at high compression ratio when compared to standard image compression techniques like JPEG and JPEG 2000.
Image completion for view synthesis using Markov random fields and efficient belief propagation

View synthesis is a process for generating novel views from a scene which has been recorded with a 3-D camera setup. It has important applications in 3-D post-production and 2-D to 3-D conversion. However, a central problem in the generation of novel views lies in the handling of disocclusions. Background content, which was occluded in the original view, may become unveiled in the synthesized view. This leads to missing information in the generated view which has to be filled in a visually plausible manner. We present an inpainting algorithm for disocclusion filling in synthesized views based on Markov random fields and efficient belief propagation. We compare the result to two state-of-the-art algorithms and demonstrate a significant improvement in image quality.
Design of no-reference video quality metrics with multiway partial least squares regression

No-reference video quality metrics are becoming ever more popular, as they are more useful in real-life applications compared to full-reference metrics. One way to design such metrics is by applying data analysis methods on both objectively measurable features and data from subjective testing. Partial least squares regression (PLSR) is one such method. In order to apply such methods, however, we have to temporally pool over all frames of a video, loosing valuable information about the quality variation over time. Hence, we extend the PLSR into a higher dimensional space with multiway PLSR in this contribution and thus consider video in all its dimensions. We designed a H.264/AVC bitstream no-reference video quality metric in order to verify multiway PLSR against PLSR with respect to the prediction performance. Our results show that the inclusion of the temporal dimension with multiway PLSR improves the quality prediction and its correlation with the actual quality.
Length-independent refinement of video quality metrics based on multiway data analysis

In previous publications it has been shown that no-reference video quality metrics based on a data analysis approach rather than on modeling the human visual system lead to very promising results and outperform many well-known full-reference metrics. Furthermore, the results improve when taking the temporal structure of the video sequence into account by using multiway analysis methods. This contribution shows a way of refining these multiway quality metrics in order to make them more suitable for real-life applications and maintaining the performance at the same time. Additionally, our results confirm the validity of H.264/AVC bitstream no-reference quality metrics using multiway PLSR by evaluating this concept on an additional dataset.
Artificial Cognition in Production Systems

Today's manufacturing and assembly systems have to be flexible to adapt quickly to an increasing number and variety of products, and changing market volumes. To manage these dynamics, several production concepts (e.g., flexible, reconfigurable, changeable or autonomous manufacturing and assembly systems) were proposed and partly realized in the past years. This paper presents the general principles of autonomy and the proposed concepts, methods and technologies to realize cognitive planning, cognitive control and cognitive operation of production systems. Starting with an introduction on the historical context of different paradigms of production (e.g., evolution of production and planning systems), different approaches for the design, planning, and operation of production systems are lined out and future trends towards fully autonomous components of an production system as well as autonomous parts and products are discussed. In flexible production systems with manual and automatic assembly tasks, human-robot cooperation is an opportunity for an ergonomic and economic manufacturing system especially for low lot sizes. The state-of-the-art and a cognitive approach in this area are outlined. Furthermore, introducing self-optimizing and self-learning control systems is a crucial factor for cognitive systems. This principles are demonstrated by a quality assurance and process control in laser welding that is used to perform improved quality monitoring. Finally, as the integration of human workers into the workflow of a production system is of the highest priority for an efficient production, worker guidance systems for manual assembly with environmentally and situationally dependent triggered paths on state-based graphs are described in this paper.