nils-herrmann/clean_scirepeval huggingface dataset:
Keypoint encoding and transmission for improved feature extraction from compressed images

In many mobile visual analysis scenarios, compressed images are transmitted over a communication network for analysis at a server. Often, the processing at the server includes some form of feature extraction and matching. Image compression has been shown to have an adverse effect on feature matching performance. To address this issue, we propose to signal the feature keypoints as side information to the server, and extract only the feature descriptors from the compressed images. To this end, we propose an approach to efficiently encode the locations, scales, and orientations of keypoints extracted from the original image. Furthermore, we propose a new approach for selecting relevant yet fragile keypoints as side information for the image, thus further reducing the data volume. We evaluate the performance of our approach using the Stanford mobile augmented reality dataset. Results show that the feature matching performance is significantly improved for images at low bitrate.
Adaptive resource allocation and frame scheduling for wireless multi-user video streaming

We propose an application-driven multi-user resource allocation and frame scheduling concept for wireless video streaming. Our approach is based on joint optimization of the application layer, the data link layer and the physical layer. For this, key parameters from these three layers are abstracted. The abstracted parameters at the application layer describe the rate-distortion characteristics of the pre-encoded video streams. At the lower layers they describe the current transmission characteristics of all users. The outcome of the joint optimization leads to adaptive resource allocation at the lower layers and an adaptive decision on which frames to send on the application layer. We show that for our scenario the expected video quality at the client side can be described analytically which leads to low complexity joint optimization. The performance of our approach is demonstrated using a real-time testbed implementation.
Dynamic segment based proxy caching for Video on Demand

In this paper, we propose a novel proxy caching scheme for video on demand (VoD) services. Our approach is based on an observation we have made during subjective VoD performance evaluation tests. We have found that when users are seeking for some specific content, they pay most attention to the initial delay, while a small shift of the starting point is acceptable. Meanwhile, the random access using the slide bar sometimes even makes this small shift unnoticeable. Based on this fact as well as the popularity of video segments, we propose a dynamic segment based caching algorithm. It adjusts the segment size according to its hitting rate and maximizes the user satisfaction by optimally trading off between the initial delay and the deviation of the starting point. Our experimental results show a significantly improved user satisfaction compared to previous caching approaches.
Analyzing LiDAR scan skewing and its impact on scan matching

This paper presents a systematic study of the scan skewing problem. Scan skewing is the non-rigid deformation of point clouds acquired by LiDAR's and is the result of their sequential scanning nature. We theoretically analyze the impact of skewing on scan matching and subsequently quantify the impact using synthetic LiDAR data with controlled skew distortions. We also show how the Geometric-Algebra LMS, an iterative point set registration filter, can be tuned to incorporate skewing aware weights to reduce the impact of skewing on scan matching. Results with real 3D LiDAR data are also presented.
Efficient Map Compression for Collaborative Visual SLAM

Swarm robotics is receiving increasing interest, because the collaborative completion of tasks, such as the exploration of unknown environments, leads to improved performance and reduced effort. The ability to exchange map information is an essential requirement for collaborative exploration. When moving to large-scale environments, where the communication data rate between the swarm participants is typically limited, efficient compression algorithms and an approach for discarding less informative parts of the map are key for a successful long-term operation. In this paper, we present a novel compression approach for environment maps obtained from a visual SLAM system. We apply feature coding to the visual information to compress the map efficiently. We make use of a minimum spanning tree to connect all features that serve as observations of a single map point. Thereby, we can exploit inter-feature dependencies and obtain an optimal coding order. Additionally, we add a map sparsification step to keep only useful map points by solving a linear integer programming problem, which preserves the map points that exhibit both good compression properties and high observability. We evaluate the proposed method on a standard dataset and show that our approach outperforms state-of-the-art techniques.
Proxy-based error tracking for H.264 based real-time video transmission in mobile environments

Error tracking (ET) is an error resilience technique for real-time video transmission over error-prone communication channels. In this paper we propose proxy-based ET for communication scenarios where the sender is in the wired Internet and the receiver is connected via a wireless link. Our main assumptions in this work are that there is a strong imbalance between the transmission rates available in the wired and the wireless Internet and that the round-trip delay is mainly caused by the wired part of the connection. We also assume that the majority of the packet loss is caused by the wireless link. In order to allow the proxy server to perform error tracking, an additional update stream is sent through the wired network. This additional information is used by the proxy to improve the performance on the wireless link We show that under these assumptions proxy-based error tracking leads to significantly improved performance for H.264 based real-time video communication in comparison to traditional end-to-end error tracking
Transparent Data Reduction in Networked Telepresence and Teleaction Systems. Part I: Communication without Time Delay

Limited communication resources represent a major challenge in networked tele-presence and teleaction systems. Video and audio compression schemes are well advanced employing models of human perception. In contrast to that haptic data reduction schemes are rather poorly treated in the known literature. This article introduces a novel approach to reduce network traffic in haptic telepresence systems exploiting limits in human haptic perception. With the proposed deadband control approach, data packets are transmitted only if the signal change exceeds a signal amplitude dependent perception threshold. Experimental user studies show that an average network traffic reduction of up to 85 can be achieved without significantly impairing the perception of the remote environment. The assumption throughout this article is that there is no communication time delay.
Are Today's Video Communication Solutions Ready for the Tactile Internet?

The Tactile Internet defines novel requirements for mobile communication, most notably ultra-low end-to-end delay. Sensors, actuators, processing pipelines, protocol stacks and display devices need to undergo major changes to comply with the radical latency requirements. In this paper, we show that today's video communication solutions, including cameras, codecs and displays are not yet ready for the Tactile Internet. We measure a glass-to-glass (G2G) delay between 120 and 386 milliseconds for commercially available video solutions for conversational services and roughly 50 milliseconds for augmented reality and teleoperation applications. Besides the analysis of existing video communication solutions we describe the delay behaviour of our prototype ultra-low-delay video system, which achieves a mean G2G delay of 19.18 milliseconds, including encoding and decoding. This is achieved with off the shelf hardware. As a second contribution, for the development of future low delay video transmission systems, we present an easy-to-use G2G delay measurement application for Android devices. The application connects to an inexpensive, highly precise measurement system via bluetooth, guides the user through the measurement process and enables data post processing and sharing.
Establishing multimodal telepresence sessions using the Session Initiation Protocol (SIP) and advanced haptic codecs

In telepresence and telemanipulation systems, multimodal data is exchanged over a network allowing humans to experience and to operate in remote or inaccessible environments. To operate over the global Internet and connect to multiple telepresence systems, a flexible framework for initiating, handling and terminating Internet-based telerobotic sessions becomes necessary. In this work, we explore the use of standard Internet session and transport protocols in the context of telerobotic applications. Session Initiation Protocol (SIP) is widely used to handle multimedia teleconference sessions with audio, video or text, and provides many services advantageous for establishing connections between heterogeneous haptic interfaces and telerobotic systems. We apply the session paradigm to the creation and negotiation of haptic telepresence sessions and propose to extend this framework to work with the haptic modality. The notion of a “haptic codec” is introduced for transforming haptic data into a common format, applying data reduction or compression techniques and implementing teleoperation control architectures. The use of the Real-Time Transport Protocol (RTP) is explored for transport of teleoperation data. Finally, a prototype and demonstrator system is presented for evaluation of the proposed framework.
Comparison of Physical-Layer and Application-Layer Diversity for Video Streaming Over Wireless Rayleigh Fading Channels

In this paper, we compare the performance of the diversity approach at the application layer and that at the physical layer for H.264/AVC encoded streaming video transmission in a wireless system with two independent channels. The application layer diversity is achieved by using multiple description source coding, while the physical layer diversity is obtained by channel coding over the two channels. Numerical results show that PHY layer diversity always outperforms the application layer diversity.
Transparency analysis of client-server-based multi-rate haptic interaction with deformable objects

In this paper we describe a client-server architecture for haptic interaction with simulated deformable objects. The computationally expensive object deformation is computed on the server at a low temporal update rate and transmitted to the clients. There, an intermediate representation of the deformable object is used to locally render haptic force feedback displayed to the user at the required rate of 1 kHz. Based on a one-dimensional deformable object, we analyze the transparency of this multi-rate architecture for a single client interaction. The delay introduced by the deformation simulation and the client-server communication leads to increased rendered forces at the clients compared to a reference scenario without delay. We propose a method that adaptively adjusts the stiffness used in the local force rendering at the client to compensate for this. The evaluation shows that the proposed method successfully compensates the effect of delay in the tested delay range of up to 100 ms.
A candidate hardware and software reference setup for kinesthetic codec standardization

Recently, the IEEE P1918.1 standardization activity has been initiated for defining a framework for the Tactile Internet. Within this activity, IEEE P1918.1.1 is a task group for the standardization of Haptic Codecs for the Tactile Internet. Primary goal of the task group is to define/develop codecs for both closed-loop (kinesthetic information exchange) and open-loop (tactile information exchange) communications. In this paper, we propose a reference hardware and software setup for the evaluation of kinesthetic codecs. The setup defines a typical teleoperation scenario in a virtual environment for the realization of closed-loop kinesthetic interactions. For the installation and testing of the setup, we provide detailed guidelines in the paper. The paper also provides sample data traces for both static and dynamic kinesthetic interactions. These data traces may be used for preliminary testing of kinesthetic codecs. The paper also provides links for the download of both the reference setup and the data traces.
Illumination-invariant image-based novelty detection in a cognitive mobile robot's environment

Image-based scene representations enable a mobile robot to make a realistic prediction of its environment. Hence, it is able to rapidly detect changes in its surroundings by comparing a virtual image generated from previously acquired reference images and its current observation. This facilitates attentional control to novel events. However, illumination effects can impair attentional control if the robot does not take them into account. To address this issue, we present in this paper an approach for the acquisition of illumination-invariant scene representations. Using multiple spatial image sequences which are captured under varying illumination conditions the robot computes an illumination-invariant image-based environment model. With this representation and statistical models about the illumination behavior, the robot is able to robustly detect texture changes in its environment under different lighting. Experimental results show high-quality images which are free of illumination effects as well as more robust novelty detection compared to state-of-the-art methods.
Haptic data reduction for time-delayed teleoperation using the time domain passivity approach

We propose a perceptual haptic data reduction approach for teleoperation systems which use the time domain passivity approach (TDPA) as their control architecture for dealing with time-varying communication delay. Our goal is to reduce the packet rate over the communication network while preserving system stability in the presence of time-varying and unknown delays. Compared to the existing wave variable-based (WV-based) haptic data reduction approaches, our proposed scheme leads to smaller distortion in the force signals and robustly deals with time-varying delays. Experiments show that our proposed approach can reduce the average packet rate by up to 80%, without introducing significant distortion. In addition, the proposed approach outperforms the existing WV-based approaches in both packet rate reduction and subjective preference for the tested communication delays.
Multi-hypothesis, volumetric reconstruction of 3-D objects from multiple calibrated camera views

In this paper we present a volumetric method for the 3-D reconstruction of real world objects from multiple calibrated camera views. The representation of the objects is fully volume-based and no explicit surface description is needed. The approach is based on multi-hypothesis tests of the voxel model back-projected into the image planes. All camera views are incorporated in the reconstruction process simultaneously and no explicit data fusion is needed. In a first step each voxel of the viewing volume is filled with several color hypotheses originating from different camera views. This leads to an overcomplete representation of the 3-D object and each voxel typically contains multiple hypotheses. In a second step only those hypotheses remain in the voxels which are consistent with all camera views where the voxel is visible. Voxels without a valid hypothesis are considered to be transparent. The methodology of our approach combines the advantages of silhouette-based and image feature-based methods. Experimental results on real and synthetic image data show the excellent visual quality of the voxel-based 3-D reconstruction.
A comparative study of network transport protocols for in-vehicle media streaming

We analyze and compare various transport protocols in the context of wireless in-vehicle IP-based audio and video communication. We determine the most appropriate transport protocol and discuss its benefits for an application in the car. The analyses are accomplished based on the IEEE 802.11 standard. A testbed is used to measure and compare quality of service values such as throughput, jitter and media quality at the receiver. In the experiments, the traditional protocols TCP and UDP showed the best performance.
Visuo-haptic sensor for force measurement and contact shape estimation

Systems interacting with objects in unstructured environments require both haptic and visual sensors to acquire sufficient scene knowledge. Typically, separate sensors and processing systems are used for the two modalities. We propose to acquire haptic and visual measurements simultaneously, with the same standard camera that already observes the scene. The compression of a passive, deformable foam rod mounted on the actuator is measured visually, yielding a 1D stress function sampled along the contour of the rod. Like that, visual and haptic measurements are naturally coherent, the system is passively compliant and the complexity of the sensor subsystems is reduced. The approach is implemented and validated on a small robotic platform which is used for haptic exploration and interacts with objects by pushing them. In our experiments, friction forces and contact points are determined when the robot comes into contact with one or multiple obstacles. Furthermore, geometric primitives of their footprints are estimated. Using a simple exploration scheme, a haptic map of static and movable objects is built.
Perception-Based Traffic Control for Shared Haptic Virtual Environments

Shared Haptic Virtual Environments (SHVEs) are often realized using a client–server communication architecture. In this case, a centralized physics engine, running on the server, is used to simulate the object-states in the virtual environment (VE). At the clients, a copy of the VE is maintained and used to render the interaction forces locally, which are then displayed to the human through a haptic device. While this architecture ensures stability in the coupling between the haptic device and the virtual environment, it necessitates a high number of object-state update packets transmitted from the server to the clients to achieve satisfactory force feedback quality. In this paper, we propose a perception-based traffic control scheme to reduce the number of object-state update packets by allowing a variable but not perceivable object-state error at the client. To find a balance between packet rate reduction and force rendering fidelity, our approach uses different error thresholds for the visual and haptic modality, where the haptic thresholds are determined by psychophysical experiments in this paper. Force feedback quality is evaluated with subjective tests for a variety of different traffic control parameter settings. The results show that the proposed scheme reduces the packet rate by up to 97%, compared to communication approaches that work without data reduction. At the same time, the proposed scheme does not degrade the haptic feedback quality significantly. Finally, it outperforms well-known dead reckoning, commonly used in visual-only distributed applications.
A SOA-based middleware concept for in-vehicle service discovery and device integration

We present a novel middleware approach for in-vehicle service discovery and device integration that is based on the concept of Service-oriented Architectures (SOA). In order to be able to identify a suitable SOA system, we define a series of criteria the SOA platform should fulfill. These criteria take into account the specific requirements of the automotive domain. Based on these criteria we compare nine different SOA standards and show why the Device Profile for Web Services (DPWS) standard is most suitable as the middleware of an in-car infotainment and communication system. Furthermore, we present a prototypical implementation of an in-vehicle network based on the DPWS middleware. Hence, we enable the integration of nomadic devices by providing their functionalities within the network and consequently including their graphical output into the in-vehicle Human Machine Interface (HMI). Due to the system design it is also possible to use nomadic devices as an additional display to present the HMI. Finally, an in-vehicle controller is included into our SOA and therefore available to all other devices within our system.
Progressive rendering from RDTC optimized streams

Recently, we have introduced a framework for the compression and interactive streaming of image-based scene representations based on hybrid video coding techniques. In that scheme, the storage rate (R), distortion (D), transmission rate (T), and decoding complexity (C) are traded off against each other to minimize the user perceived delay in a remote navigation scenario. In this work, we extend that scheme by introducing a technique for progressive transmission and rendering from such RDTC compressed image-based scene representations. Real-time experiments show that a trade off between the user perceived delay and the rendering quality can be achieved and that the delay can be significantly reduced compared to independent or rate-distortion optimized compression.
Quality-of-control-driven uplink scheduling for networked control systems running over 5G communication networks

The emerging 5G networks will enable control-oriented applications where sensors, actuators, processors and controllers are geographically distributed and wirelessly connected. One major challenge of networked control systems (NCSs) is how to guarantee the stability of all control loops in the presence of communication delay. Addressing this problem, the control community generally focuses on designing new controllers or adjusting the sensor's sampling period to adapt to the network. Alternatively, we address the problem from the communication perspective in this paper. To this end, we consider all control loops as network applications (i.e. keeping controllers, sensors and actuators unchanged), and develop a control-aware network scheduler to handle the control performance degradation caused by communication delay. We further investigate how to interpret the Quality-of-Control as a Quality-of-Service (QoS) measure to guide the scheduler design. The proposed QoS metric represents the instantaneous Linear Quadratic Gaussian (LQG) cost of a control system, which quantifies the energy deviation of the system to its optimal condition. By leveraging the proposed QoS metric, we develop a novel QoS-driven uplink scheduler. Simulation results confirm the efficiency of the proposed scheduler for networked control services.
ConCor+: Robust and confident video synchronization using consensus-based Cross-correlation

Consensus-based Cross-correlation (ConCor) is a recently presented algorithm for robust synchronization of noisy and corrupted signals. ConCor has a number of interdependent parameters that need to be set correctly to guarantee good performance. In this paper we analyse the effects of the individual parameters on ConCor's behaviour and performance. As a second contribution, we show that a parameter sweep with subsequent majority voting can be used to boost ConCor's performance and produce a trustworthy confidence measure. As a final contribution we show how the proposed extension also allows performing multi-modal (joint audio-video) synchronization of casual multi-perspective video recordings enabling superior matching performance.