nils-herrmann/clean_scirepeval huggingface dataset:
Distributed Collaboration for Enhanced Sender-Driven Video Streaming

We propose a sender-driven system for adaptive streaming from multiple servers to a single receiver over separate network paths. The servers employ information in receiver feedbacks to estimate the available bandwidth on the paths and then compute appropriate transmission schedules for streaming media packets to the receiver based on the bandwidth estimates. An optimization framework is proposed that enables the senders to compute their transmission schedules in a distributed way, and yet to dynamically coordinate them over time such that the resulting video quality at the receiver is maximized. To reduce the computational complexity of the optimization framework an alternative technique based on packet classification is proposed. The substantial reduction in online complexity due to the resulting packet partitioning makes the technique suitable for practical implementations of adaptive and efficient distributed streaming systems. Simulations with Internet network traces demonstrate that the proposed solution adapts effectively to bandwidth variations and packet loss. They show that the proposed streaming framework provides superior performance over a conventional distortion-agnostic scheme that performs proportional packet scheduling on the network paths according to their respective bandwidth values.
Dense disparity estimation from omnidirectional images

This paper addresses the problem of dense estimation of disparities between omnidirectional images, in a spherical framework. Omnidirectional imaging certainly represents important advantages for the representation and processing of the plenoptic function in 3D scenes for applications in localization, or depth estimation for example. In this context, we propose to perform disparity estimation directly in a spherical framework, in order to avoid discrepancies due to inexact projections of omnidirectional images onto planes. We first perform rectification of the omnidirectional images in the spherical domain. Then we develop a global energy minimization algorithm based on the graph-cut algorithm, in order to perform disparity estimation on the sphere. Experimental results show that the proposed algorithm outperforms typical methods as the ones based on block matching, for both a simple synthetic scene, and complex natural scenes. The proposed method shows promising performances for dense disparity estimation and can be extended efficiently to networks of several camera sensors.
Bit rate allocation for disparity estimation from compressed images

This paper presents a novel rate allocation scheme to compute the 3D structure of the scene from compressed stereo images, captured by the distributed vision sensor networks. The images captured at different view points are encoded independently with a balanced rate allocation. The central decoder jointly decodes the information from the encoders, and computes the 3D geometry of the scene in terms of depth map. We first consider the scenario of estimating the 3D geometry from the views, compressed using standard encoders, e.g., SPIHT. Unfortunately, we noticed that the depth value is not precisely reconstructed in the low contrast regions or region around weak edges. It is mainly due to the rate allocation scheme, that allocates the bits based on the variance of the coefficients. We therefore propose a rate allocation scheme, where each encoder first identifies the low contrast regions and then distributes the bits such that the visual information in the low contrast regions is preserved. At the same time, the approximation quality in the rest of the image should not be penalized significantly. We adapt the SPIHT coding scheme to implement the proposed rate allocation methodology. Experimental results show that for a given bit budget, the proposed encoding scheme reconstructs the 3D geometry with more accuracy comparing to SPIHT, JPEG 2000 and JPEG coding schemes.
Decoding Delay Minimization in Inter-Session Network Coding

Intra-session network coding has been shown to offer significant gains in terms of achievable throughput and delay in settings where one source multicasts data to several clients. In this paper, we consider a more general scenario where multiple sources transmit data to sets of clients over a wireline overlay network. We propose a novel framework for efficient rate allocation in networks where intermediate network nodes have the opportunity to combine packets from different sources using randomized network coding. We formulate the problem as the minimization of the average decoding delay in the client population and solve it with a gradient-based stochastic algorithm. Our optimized inter-session network coding solution is evaluated in different network topologies and is compared with basic intra-session network coding solutions. Our results show the benefits of proper coding decisions and effective rate allocation for lowering the decoding delay when the network is used by concurrent multicast sessions.
Distortion-buffer optimized TCP video streaming

This paper presents a distortion optimized streaming algorithm for on-demand streaming of multimedia. Given the pre-encoded packets of a multimedia stream, we propose an algorithm for selecting an appropriate subset of these packets such that the overall client distortion is minimized. This minimization is performed within the rate constraints imposed by the communication channel. In the interest of computation it is desirable to limit the horizon (i.e. the look-ahead) over which the optimization is performed. Inevitably, shortening the horizon leads to sub-optimal results. We alleviate the impact due to this through the introduction of a buffering constraint that stipulates a minimum desired buffer occupancy at all time during the streaming session. We pose this problem as a Lagrangian minimization - the solution to which is obtained through an iterative descent algorithm. We demonstrate the efficacy of the proposed approach through empirical evaluation.
Geometry-driven quantization for omnidirectional image coding

In this paper we propose a method to adapt the quantization tables of typical block-based transform codecs when the input to the encoder is a panoramic image resulting from equirectangular projection of a spherical image. When the visual content is projected from the panorama to the viewport, a frequency shift is occurring. The quantization can be adapted accordingly: the quantization step sizes that would be optimal to quantize the transform coefficients of the viewport image block, can be used to quantize the coefficients of the panoramic block. As a proof of concept, the proposed quantization strategy has been used in JPEG compression. Results show that a rate reduction up to 2.99% can be achieved for the same perceptual quality of the spherical signal with respect to a standard quantization.
OmniSIFT: Scale invariant features in omnidirectional images

We propose a method to compute scale invariant features in omnidirectional images. We present a formulation based on Riemannian geometry for the definition of differential operators on non-Euclidian manifolds that correspond to the particular form of the mirrors in omnidirectional imaging. These operators lead to a scale-space analysis that preserves the geometry of the visual information in omnidirectional images. We eventually build novel scale-invariant omniSIFT features inspired by the planar SIFT framework. We apply our generic solution to omnidirectional images captured with parabolic mirrors. Simple descriptors that use omniSIFT characteristics offer promising performance in the case of image rotation or translation where visual features can be preserved due to the proper handling of the implicit image geometry.
Price-Based Controller for Utility-Aware HTTP Adaptive Streaming

HTTP Adaptive Streaming (HAS) can efficiently deliver video to multiple heterogeneous users in a fully distributed way. This might, however, lead to unfair bandwidth utilization among HAS users. Therefore, network-assisted HAS systems have been proposed, where network elements operate alongside clients' adaptation logic to improve user satisfaction. However, current solutions rely on the assumption that network elements have full knowledge of the network status, which isn't always realistic. In this work, the authors propose a practical network-assisted HAS system where the network elements infer the network link congestion using measurements collected from the client endpoints. The congestion-level signal is then used by the clients to optimize their video data requests. The authors' novel controller maximizes overall user satisfaction, and the clients share the available bandwidth fairly from a utility perspective, as demonstrated by simulation results obtained on a network simulator.
Graph-based vs depth-based data representation for multiview images

In this paper, we propose a representation and coding method for multiview images. As an alternative to depth-based schemes, we propose a representation that captures the geometry and the dependencies between pixels in different views in the form of connections in a graph. In our approach it is possible to perform compression of the geometry information and to preserve a direct control of the effect of geometry approximation on view reconstruction. This is not possible with classical depth-based representations. As a results, our method leads to more accurate view prediction, when compared to conventional lossy coding of depth maps operating at the same bit rate. We finally show in experiments that our representation adapts the amount of transmitted geometry to the complexity of the predictions that are performed at the decoder.
Semantic Coding by Supervised Dimensionality Reduction

This paper addresses the problem of representing multimedia information under a compressed form that permits efficient classification. The semantic coding problem starts from a subspace method where dimensionality reduction is formulated as a matrix factorization problem. Data samples are jointly represented in a common subspace extracted from a redundant dictionary of basis functions. We first build on greedy pursuit algorithms for simultaneous sparse approximations to solve the dimensionality reduction problem. The method is extended into a supervised algorithm, which further encourages the class separability in the extraction of the most relevant features. The resulting supervised dimensionality reduction scheme provides an interesting tradeoff between approximation (or compression) and discriminant feature extraction (or classification). The algorithm provides a compressed signal representation that can directly be used for multimedia data mining. The application of the proposed algorithm to image recognition problems further demonstrates classification performances that are competitive with state-of-the-art solutions in handwritten digit or face recognition. Semantic coding certainly represents an interesting solution to the challenging problem of processing huge volumes of multidimensional data in modern multimedia systems, where compressed data have to be processed and analyzed with limited computational complexity.
Seismic Simultaneous Source Separation via Patchwise Sparse Representation

The concept of simultaneous source has recently become of interest in seismic exploration, due to its efficient or economic acquisition or both. The blended data overlapped between shot records are acquired in simultaneous source acquisition. Separating the blended data and recovering the single-shot seismic signals (the recovery) are of great importance in the scenario of current workflows, which can be called seismic simultaneous source separation. In the context of general random time-dithering firing, we propose an alternative method to separate the blended data by combining patchwise dictionary learning with sparse inversion, in which the dictionary is directly learned from the measured blended data. Apart from the sparse coding used for the coefficients, an additional regularization term on the dictionary is particularly designed to remove the severe interference noise. The efficient and flexible alternating direction method of multipliers (ADMM) is used to update the dictionary in the used alternating optimization scheme. The results obtained from the synthetic and real examples reasonably suggest that the separated seismic signals by using dictionary learning are more accurate and robust compared with that using the fixed transform basis, such as the local discrete cosine transform. The learned dictionary tailors for the recovery and is similar to the local seismic waveform, which improves the sparsity of the recovery substantially and is highly advantageous for producing the promised results.
Markov Decision Process Based Energy-Efficient On-Line Scheduling for Slice-Parallel Video Decoders on Multicore Systems

We consider the problem of energy-efficient on-line scheduling for slice-parallel video decoders on multicore systems with Dynamic Voltage Frequency Scaling (DVFS) enabled processors. In the past, scheduling and DVFS policies in multi-core systems have been formulated heuristically due to the inherent complexity of the on-line multicore scheduling problem. The key contribution of this paper is that we rigorously formulate the problem as a Markov decision process (MDP), which simultaneously takes into account the on-line scheduling and per-core DVFS capabilities; the power consumption of the processor cores and caches; and the loss tolerant and dynamic nature of the video decoder. The objective of the MDP is to minimize long-term power consumption subject to a minimum Quality of Service (QoS) constraint related to the decoder's throughput. We evaluate the proposed on-line scheduling algorithm in Matlab using realistic video decoding traces generated from a cycle-accurate multiprocessor ARM simulator.
Reference View Selection in DIBR-Based Multiview Coding

Augmented reality, interactive navigation in 3D scenes, multiview video, and other emerging multimedia applications require large sets of images, hence larger data volumes and increased resources compared with traditional video services. The significant increase in the number of images in multiview systems leads to new challenging problems in data representation and data transmission to provide high quality of experience on resource-constrained environments. In order to reduce the size of the data, different multiview video compression strategies have been proposed recently. Most of them use the concept of reference or key views that are used to estimate other images when there is high correlation in the data set. In such coding schemes, the two following questions become fundamental: 1) how many reference views have to be chosen for keeping a good reconstruction quality under coding cost constraints? And 2) where to place these key views in the multiview data set? As these questions are largely overlooked in the literature, we study the reference view selection problem and propose an algorithm for the optimal selection of reference views in multiview coding systems. Based on a novel metric that measures the similarity between the views, we formulate an optimization problem for the positioning of the reference views, such that both the distortion of the view reconstruction and the coding rate cost are minimized. We solve this new problem with a shortest path algorithm that determines both the optimal number of reference views and their positions in the image set. We experimentally validate our solution in a practical multiview distributed coding system and in the standardized 3D-HEVC multiview coding scheme. We show that considering the 3D scene geometry in the reference view, positioning problem brings significant rate-distortion improvements and outperforms the traditional coding strategy that simply selects key frames based on the distance between cameras.
Optimal polynomial filtering for accelerating distributed consensus

In the past few years, the problem of distributed consensus has received a lot of attention, particularly in the framework of ad hoc sensor networks. Most methods proposed in the literature attack this problem by distributed linear iterative algorithms, with asymptotic convergence of the consensus solution. It is known that the rate of convergence depends on the second largest eigenvalue of the weight matrix. In this paper, we propose the use of polynomial filtering in order to accelerate the convergence rate. The main idea of the proposed methodology is to apply a polynomial filter that will shape the spectrum of the weight matrix by minimizing its second largest eigenvalue and therefore increase the convergence rate. We formulate the computation of the optimal polynomial as a semi-definite program (SDP) that can be efficiently and globally solved. We provide simulation results that demonstrate the validity and effectiveness of the proposed scheme in both fixed and dynamic network topologies.
A posteriori quantization of progressive matching pursuit streams

This paper proposes a rate-distortion optimal a posteriori quantization scheme for matching pursuit (MP) coefficients. The a posteriori quantization applies to an MP expansion that has been generated offline and cannot benefit of any feedback loop to the encoder in order to compensate for the quantization noise. The redundancy of the MP dictionary provides an indicator of the relative importance of coefficients and atom indices and, subsequently, on the quantization error. It is used to define a universal upper bound on the decay of the coefficients, sorted in decreasing order of magnitude. A new quantization scheme is then derived, where this bound is used as an Oracle for the design of an optimal a posteriori quantizer. The latter turns the exponentially distributed coefficient entropy-constrained quantization problem into a simple uniform quantization problem. Using simulations with random dictionaries, we show that the proposed exponentially upper bounded quantization (EUQ) clearly outperforms classical schemes. Stepping on the ideal Oracle-based approach, a suboptimal adaptive scheme is then designed that approximates the EUQ but still outperforms competing quantization methods in terms of rate-distortion characteristics. Finally, the proposed quantization method is studied in the context of image coding. It performs similarly to state-of-the-art coding methods (and even better at low rates) while interestingly providing a progressive stream that is very easy to transcode and adapt to changing rate constraints.
An adaptive cross layer resource allocation scheme for correlated wireless video sources

In this work we study adaptive resource allocation for uplink transmission of correlated video sources. We consider a framework where multiple wireless sources transmit correlated information via a common base station. We introduce an optimization problem for sources to maximize the weighted sum of received quality of all videos, when source correlation can be used at the decoder in case of missing data. Each source finds its respective Multiple Access (MAC) parameters and performs packet selection. This is done with minimal information exchange with the base station. We model the quality of a decoded video as a piecewise linear function of qualities of the most correlated views, and verify the validity of the piecewise linear quality model for a two source case. We use this model to simplify the resource allocation method. We then compare performance of our correlated resource allocation to optimal resource allocation for independent sources, as well as to a baseline method. The simulations show that our proposed method results in higher average Y-PSNR than the optimal independent resource allocation in most channel conditions, without significative complexity increase in the base station or the source nodes.
The Robustness of Deep Networks: A Geometrical Perspective

Deep neural networks have recently shown impressive classification performance on a diverse set of visual tasks. When deployed in real-world (noise-prone) environments, it is equally important that these classifiers satisfy robustness guarantees: small perturbations applied to the samples should not yield significant loss to the performance of the predictor. The goal of this article is to discuss the robustness of deep networks to a diverse set of perturbations that may affect the samples in practice, including adversarial perturbations, random noise, and geometric transformations. This article further discusses the recent works that build on the robustness analysis to provide geometric insights on the classifier's decision surface, which help in developing a better understanding of deep networks. Finally, we present recent solutions that attempt to increase the robustness of deep networks. We hope this review article will contribute to shed ding light on the open research challenges in the robustness of deep networks and stir interest in the analysis of their fundamental properties.
Graph-Based Classification of Omnidirectional Images

Omnidirectional cameras are widely used in such areas as robotics and virtual reality as they provide a wide field of view. Their images are often processed with classical methods, which might unfortunately lead to non-optimal solutions as these methods are designed for planar images that have different geometrical properties than omnidirectional ones. In this paper we study image classification task by taking into account the specific geometry of omnidirectional cameras with graph-based representations. In particular, we extend deep learning architectures to data on graphs; we propose a principled way of graph construction such that convolutional filters respond similarly for the same pattern on different positions of the image regardless of lens distortions. Our experiments show that the proposed method outperforms current techniques for the omnidirectional image classification problem.
NC node selection game in collaborative streaming systems

Network coding has been recently proposed as an efficient method to improve throughput, minimize delays and remove the need for reconciliation between network nodes in distributed streaming systems. It permits to take advantage of the path and node diversity in the network when the network coding nodes are placed efficiently. In this paper, we investigate networks consisting of nodes that autonomously determine whether they should perform network coding or not as well as their set of parent nodes. Each node makes its decisions that maximize its quality of service. The decisions include the selection of operation mode (i.e., network coding mode, simple data forwarding mode) and the selection of extra connections. The resulting interactions among the nodes are modeled as a congestion game, thereby ensuring an equilibrium, i.e., stable multimedia stream flow. The experimental results show that the proposed scheme is appropriate for distributed multimedia transmission since it provides a stable quality without imposing centralized control.