nils-herrmann/clean_scirepeval huggingface dataset:
Passivity-based model updating for Model-mediated Teleoperation

We study the stability of three-dimensional (3D) haptic interaction in Model-mediated Teleoperation (MMT) systems. In MMT, a simple object model is employed on the master side to approximate the remote environment. The haptic feedback is rendered locally on the master side based on model parameters which are estimated on the slave side. The main advantage of the MMT approach is that the haptic control loop is running locally which leads to stable behavior even in the presence of communication delays. The local model must be updated when the environment changes, or novel, previously unseen parts are encountered. During the model update, a sudden change of the model parameters leads to model-jump effects and results in unpredictable motion and force. A smooth and stable model update scheme is required to mitigate this effect. To guarantee system stability, we derive a passivity condition for a 3D spring-damper model during the model update phase. Energy generated due to the changes of the model parameters is dissipated through an adaptive damper element. The sampling effect of the haptic device is also taken into account. The performance of the proposed passivity-based model update (PMU) scheme is evaluated through simulations and subjective experiments. Evaluation results show that by using the PMU scheme, system stability during the model update phase is guaranteed. Moreover, subjects feel more comfortable with the model update if the PMU scheme is enabled.
Prediction techniques for haptic communication and their vulnerability to packet losses

We introduce three different uses of linear regression for improving the prediction of samples in haptic communication and compare this technique to the commonly employed zero-order and first-order linear predictors. We couple the prediction techniques with (error resilient) perceptual data reduction approaches and evaluate their robustness when losses in the network are present. Experimental results show that the proposed prediction technique improves haptic data reduction while keeping lower signal distortion compared to the traditional prediction methods when facing adverse network conditions.
Grasping posture estimation for a two-finger parallel gripper with soft material jaws using a curved contact area friction model

We present a friction model for the curved contact area between a deformable object and soft parallel gripper jaws for grasping posture estimation. We show that the assumption of a planar contact area leads to an overestimation of the frictional force and torque, which might cause the object to slip. We simulate the contact with the Finite Element Method, then compute the friction wrenches, which are fitted with two limit surface models: an ellipsoid and a convex 4th-order polynomial. Despite a slightly higher fitting error, the ellipsoid limit surface is chosen to compute the grasp quality because of its simplicity. We compare the limit surfaces of our friction model with the planar contact model and show the improved accuracy obtainable with our model. We then apply the presented model for grasping posture estimation by simulating the contact for all grasp candidates. We show a grasp quality map (quality of all grasp candidates) and the best possible grasp location for several deformable objects.
Multi-rate encoding for HEVC-based adaptive HTTP streaming with multiple resolutions

Adaptive HTTP streaming requires a video to be encoded at different rates and qualities called representations. The encoding of multiple representations with the new video coding standard HEVC is computationally complex. In this paper, we propose a multi-rate encoding method which reduces the complexity of encoding a video at multiple spatial resolutions. We first examine block structure similarities at different resolutions and propose a method to derive the block structure for a low resolution representation from a reference high resolution encoding. The derived block structure is used to speed up the encoding of low resolution representations. We further consider the content of the videos in order to achieve a rate-distortion (RD) performance similar to independent HEVC encoding. Experimental results show that the encoding time can be reduced by 50% on average for a low resolution video without degrading the RD performance.
Perceptual and Bitrate-Scalable Coding of Haptic Surface Texture Signals

Applications involving indirect interpersonal communication, such as collaborative design/assembly/exploration of physical objects, can benefit strongly from the transmission of contact-based haptic media, in addition to the more traditional audiovisual media. Inclusion of haptic media has been shown to improve immersiveness, task performance, and the overall experience of task execution. While several decades of research have been dedicated to the acquisition, processing, coding, and display of audio and video streams, similar aspects for haptic streams have been addressed only recently. Simultaneous masking is a perceptual phenomenon widely exploited in the compression of audio data. In the first part of this paper, to the best of our knowledge, we present first-time empirical evidence for masking in the perception of wideband vibrotactile signals. Our results show that this phenomenon for haptics is very similar to its auditory analog. Signals closer in frequency to a powerful masker ( 25 dB above detection threshold) are masked more strongly (peak threshold-shifts of up to 28 dB) than those away from the masker (threshold-shifts of 15-20 dB). The masking curves approximately follow the masker's spectral profile. In the second part of this paper, we present a bitrate scalable haptic texture codec, which incorporates the masking model and describe its subjective and objective performance evaluation. Experiments show that we can drive down the codec output bitrate to a very low value of 2.3 kbps, without the subjects being able to reliable discriminate between the codec input and distorted output texture signals.
QoE-driven live and on-demand LTE uplink video transmission

We consider the joint upstreaming of live and on-demand user-generated video content over LTE using a Quality-of-Experience driven approach. We contribute to the state-of-the-art work on multimedia scheduling in three aspects: 1) we jointly optimize the transmission of live and time-shifted video under scarce uplink resources by transmitting a basic quality in realtime and uploading a refined quality for on-demand consumption. 2) We propose a producer-consumer deadline-aware scheduling algorithm that incorporates both the physical state of the mobile producer (e.g., cache fullness) and the scheduled playout time at the end-user. 3) We show that the scheduling decisions in 1) and 2) can be determined locally for each mobile producer. We additionally present an analytical framework for de-centralized scalable video transmission and prove that there exists an optimal solution to our problem. Simulation results for LTE uplink further demonstrate the significance of our proposed optimization on the overall user experience.
Noise-Resistant Deep Learning for Object Classification in Three-Dimensional Point Clouds Using a Point Pair Descriptor

Object retrieval and classification in point cloud data are challenged by noise, irregular sampling density, and occlusion. To address this issue, we propose a point pair descriptor that is robust to noise and occlusion and achieves high retrieval accuracy. We further show how the proposed descriptor can be used in a four-dimensional (4-D) convolutional neural network for the task of object classification. We propose a novel 4-D convolutional layer that is able to learn class-specific clusters in the descriptor histograms. Finally, we provide experimental validation on three benchmark datasets, which confirms the superiority of the proposed approach.
Scale-preserving long-term visual odometry for indoor navigation

We present a visual odometry system for indoor navigation with a focus on long-term robustness and consistency. As our work is targeting mobile phones, we employ monocular SLAM to jointly estimate a local map and the device's trajectory. We specifically address the problem of estimating the scale factor of both, the map and the trajectory. State-of-the-art solutions approach this problem with an Extended Kalman Filter (EKF), which estimates the scale by fusing inertial and visual data, but strongly relies on good initialization and takes time to converge. Each visual tracking failure introduces a new arbitrary scale factor, forcing the filter to re-converge. We propose a fast and robust method for scale initialization that exploits basic geometric properties of the learned local map. Using random projections, we efficiently compute geometric properties from the feature point cloud produced by the visual SLAM system. From these properties (e.g., corridor width or height) we estimate scale changes caused by tracking failures and update the EKF accordingly. As a result, previously achieved convergence is preserved despite re-initializations of the map. To minimize the time required to continue tracking after failure, we perform recovery and re-initialization in parallel. This increases the time available for recovery and hence the likelihood for success, thus allowing almost seamless tracking. Moreover, fewer re-initializations are necessary. We evaluate our approach using extensive and diverse indoor datasets. Results demonstrate that errors and convergence times for scale estimation are considerably reduced, thus ensuring consistent and accurate scale estimation. This enables long-term odometry despite of tracking failures which are inevitable in realistic scenarios.
Model-Mediated Teleoperation: Toward Stable and Transparent Teleoperation Systems

Bilateral teleoperation systems with haptic feedback allow human users to interact with objects or perform complex tasks in remote or inaccessible environments. Communication delays in teleoperation systems jeopardize system stability and transparency, leading to degraded system performance and poor user experience. In this paper, we provide a survey of the model-mediated teleoperation (MMT) approach, which has been developed to guarantee both system stability and transparency in the presence of arbitrary communication delays. This survey focuses on two major parts: 1) the historical development of the MMT approach from the late 1980s to the present and 2) the main challenges facing the design of a reliable MMT system. Along with the discussion of the MMT challenges and the proposed solutions, a series of experiments has been conducted to compare the performance between the existing techniques and to supply data that were missing in the previous studies on the MMT approach.
Towards real-time modeling and haptic rendering of deformable objects for point cloud-based Model-Mediated Teleoperation

We propose a novel radial function-based deformation (RFBD) approach to enable real-time modeling and haptic rendering of deformable objects for point cloud-based Model-Mediated Teleoperation (pcbMMT). In the pcbMMT system, a time-of-flight camera is employed on the slave side to capture a 3D point cloud model of the remote environment. This environment model is transmitted to the master along with the estimated model parameters, including the dynamics of the object's surface deformation and the physical properties such as the stiffness, friction coefficient, etc. Due to the simplicity of the RFBD approach, the model parameters of the remote environment can be obtained in real time. On the master side, a local copy of the environment model is maintained according to the received parameters. Both the haptic rendering and deformation simulation are based on this local model, running at 1kHz. Thus, a good compromise is achieved between the model accuracy and the computational time for online parameter identification. Experiments verify the feasibility of the proposed approach, and show the results of the object deformation for different model parameters.
Network-aware video level encoding for uplink adaptive HTTP streaming

We study the uplink delivery of live video using adaptive HTTP streaming (AHS). In AHS, the process of simultaneously creating video levels at different rates is computationally demanding and quickly exceeds the computational capacity of mobile devices. As a remedy, we propose a network-aware video level selection approach which reduces the number of levels that need to be encoded. To this end, we develop an algorithm which selects a reduced set of video levels from a static pre-defined set based on TCP uplink throughput information. More specifically, during session start-up and after inter-RAN handovers, TCP uplink throughput information from a remote database is used while otherwise, actual TCP uplink throughput measurements are performed. We test the proposed approach in an automotive scenario to upstream the video of a vehicle's front-facing camera to a remote video portal. Our results show that our proposed network-aware video level selection approach leads to a significant reduction of the number of video levels that need to be encoded. At the same time, a similar quality of experience is achieved in terms of mean subjective quality of the delivered video segments, interrupted playback duration due to stalling events, and number of quality switches when compared to an implementation which considers the static full set of video levels.
Haptic Data Compression and Communication

The past decade has witnessed how audio-visual communication has shaped the way humans interact with or through technical systems. In contemporary times, the potential of haptic communication has been recognized as being compelling to further augment human-to-human and human-to-machine interaction. In the context of immersive communication, video and audio compression are considered key enabling technologies for high-quality interaction. In contrast, the compression of haptic data is a field of research that is still relatively young and not fully explored. This disregards the fact that we as humans rely heavily on the haptic modality to interact with our environment. True immersion into a distant environment and efficient collaboration between multiple participants both require the ability to physically interact with objects in the remote environment. With recent advances in virtual reality, man-machine interaction, telerobotics, telepresence, and teleaction, haptic communication is proving instrumental in enabling many novel applications. The goal of this overview article is to summarize the state of the art and the challenges of haptic data compression and communication for telepresence and teleaction.
Keypoint Encoding for Improved Feature Extraction From Compressed Video at Low Bitrates

In many mobile visual analysis applications, compressed video is transmitted over a communication network and analyzed by a server. Typical processing steps performed at the server include keypoint detection, descriptor calculation, and feature matching. Video compression has been shown to have an adverse effect on feature-matching performance. The negative impact of compression can be reduced by using the keypoints extracted from the uncompressed video to calculate descriptors from the compressed video. Based on this observation, we propose to provide these keypoints to the server as side information and to extract only the descriptors from the compressed video. First, we introduce four different frame types for keypoint encoding to address different types of changes in video content. These frame types represent a new scene, the same scene, a slowly changing scene, or a rapidly moving scene, and are determined by comparing features between successive video frames. Then, we propose Intra, Skip, and Inter modes of encoding the keypoints for different frame types. For example, keypoints for new scenes are encoded using the Intra mode, and keypoints for unchanged scenes are skipped. As a result, the bitrate of the side information related to keypoint encoding is significantly reduced. Finally, we present pairwise matching and image retrieval experiments conducted to evaluate the performance of the proposed approach using the Stanford mobile augmented reality dataset and 720p format videos. The results show that the proposed approach offers significantly improved feature matching and image retrieval performance at a given bitrate.
Surprise-driven acquisition of visual object representations for cognitive mobile robots

Robots in a household environment have to execute a variety of tasks including carrying objects. In order to grasp the correct objects for a desired action it is indispensable that the robot is able to recognize the objects in its environment. From time to time, the robot will encounter new unknown objects which it has never seen before. In order to recognize them in a later task the robot has to acquire an internal representation of them. In this paper, we present an approach for the autonomous acquisition of visual object representations in a cluttered environment. Guided by surprise, the robot detects novel objects in a familiar environment, selects local image features which represent their appearance and stores them in a database. Experimental results show that our method for the detection of surprising events reliably directs the robot's attention to the novel objects and that the recognition behavior based on our acquired object representations outperforms a state-of-the-art approach.
Improved ρ-domain rate control with accurate header size estimation

ρ-domain rate control has been shown to be a simple yet effective rate control approach for DCT-based hybrid video codecs. When it is applied to H.264/AVC, improvements need to be made because of the large amount of header information and the QP-dependent Rate Distortion Optimization (RDO). In this paper, we propose a novel rate model to estimate the size of header information accurately. We present a two-stage rate control algorithm which combines the proposed header rate model and the ρ-domain source model. In comparison to the original ρ-domain rate control algorithm without header size estimation, our scheme not only improves the PSNR of the decoded video, but also achieves the target bit rates more accurately.
HTTP-based scalable video streaming over mobile networks

This paper proposes an adaptive HTTP-based video streaming framework for mobile networks using the Scalable Video Coding (SVC) extension to the H.264 standard. We present a method to statistically estimate the channel at the mobile client and use it in our work to adapt the bit rate of the video. The adaptation takes into account both the quality contributions and the probability of successful timely decoding for different video segments. The simulation results show significant improvements in terms of a reduction of playback interruptions and improved perceived quality of service.
Proxy-based reference picture selection for real-time video transmission over mobile networks

We propose a framework for error robust real-time video transmission over wireless networks. In our approach, we cope with packet loss on the downlink by retransmitting lost packets from the base station (BS) to the receiver for error recovery. Retransmissions are enabled by using fixed-distance reference picture selection during encoding with a prediction distance that corresponds to the round-trip-time between the BS and the receiver. We deal with transmission errors on the uplink by sending acknowledgements and predicting the next frame from the most recent frame that has been positively acknowledged by the BS. We show that these two separate approaches for uplink and downlink nicely fit together. We compare our approach to state-of-the art error resilience approaches that employ random intra update of macroblocks and FEC across packets for error resilience. At the same bit rate and packet loss rate we observe improvements of up to 4.5 dB for our scheme.
3-D object reconstruction using spatially extended voxels and multi-hypothesis voxel coloring

We describe a voxel-based 3-D reconstruction technique from multiple calibrated camera views that makes explicit use of the finite size footprint of a voxel when projected into the image plane. We derive a class of computationally efficient axis-aligned volume traversal orders that ensure that a processed voxel cannot occlude previously processed voxels. For each view, one out of 79 different cases of volume traversal is identified depending on the relative position between camera and voxel volume. Views belonging to the same visibility class can be processed simultaneously. Our voxel coloring strategy is based on a color hypothesis test that ensures the consistency of the projected reconstruction with the original images. A surface voxel list is constantly updated during reconstruction ensuring that only a minimum number of voxels has to be processed. Experimental results that compare the reconstruction quality for voxels with and without spatial extent underscore the conclusion that it is worthwhile taking into account the exact footprint of the projected voxels.
Application-driven cross layer optimization for wireless networks using MOS-based utility functions

This paper discusses a Quality of Experience (QoE) driven cross-layer optimization framework for efficient network resource allocation in wireless networks. The proposed scheme jointly optimizes the application layer and the lower layers of the wireless protocol stack with the aim of improving the user's QoE. The Mean Opinion Score (MOS) is used as a common metric for user-perceived quality in the optimization scheme. Three different QoE-based optimization schemes are compared to a throughput maximization scheme and a non-optimized system. We perform simulations using a software implementation of a developed HSDPA system. Results show that the MOS-based approaches lead to significant improvements of user perceived quality.
Multiple description video coding using motion-compensated lifted 3D wavelet decomposition

We design a multiple description (MD) video coding scheme based on the motion compensated (MC) lifted wavelet transform. A major advantage of basing MD video coding on motion-compensated lifted 3D wavelet decomposition is that it does not require any mismatch control as in a hybrid codec which was previously achieved by sending drift compensation data. We propose to perform the temporal decomposition of a group of pictures and then create multiple descriptions for each temporally transformed frame. We use polyphase subsampling with an appropriate amount of oversampling to form the descriptions and to control the redundancy among them. We determine the appropriate amount of redundancy as a function of description loss probability. Our experimental results show that the controlled introduction of redundancy leads to significant improvements in lossy transmission environments.
Exploiting directional dependencies of force perception for lossy haptic data reduction

Packet-switched communication of haptic data in real-time telemanipulation systems is characterized by high packet rates on the communication channel. Perceptual data reduction techniques for haptic signals successfully address this challenge. By applying a mathematical model of human haptic perception, the transmission of imperceptible signal changes over the network can be avoided. When extending existing haptic data reduction schemes from a single degree of freedom (DoF) to multi-DoFs, the spatial orientation of force feedback vectors acts as an additional perceptual dimension. In this paper, we investigate directional dependencies of haptic data reduction artifacts. Experimental studies reveal that the performance of perceptual data reduction can be significantly improved by adapting the perception thresholds to the spatial direction of force feedback.