nils-herrmann/clean_scirepeval huggingface dataset:
A VoD System for Massively Scaled, Heterogeneous Environments: Design and Implementation

We propose, analyze and implement a general architecture for massively parallel VoD content distribution. We allow for devices that have a wide range of reliability, storage and bandwidth constraints. Each device can act as a cache for other devices and can also communicate with a central server. Some devices may be dedicated caches with no co-located users. Our goal is to allow each user device to be able to stream any movie from a large catalog, while minimizing the load of the central server. First, we architect and formulate a static optimization problem that accounts for various network bandwidth and storage capacity constraints, as well as the maximum number of network connections for each device. Not surprisingly this formulation is NP-hard. We then use a Markov approximation technique in a primal-dual framework to devise a highly distributed algorithm which is provably close to the optimal. Next we test the practical effectiveness of the distributed algorithm in several ways. We demonstrate remarkable robustness to system scale and changes in demand, user churn, network failure and node failures via a packet level simulation of the system. Finally, we describe our results from numerous experiments on a full implementation of the system with 60 caches and 120 users on 20 Amazon EC2 instances. In addition to corroborating our analytical and simulation-based findings, the implementation allows us to examine various system-level tradeoffs. Examples of this include: (i) the split between server to cache and cache to device traffic, (ii) the tradeoff between cache update intervals and the time taken for the system to adjust to changes in demand, and (iii) the tradeoff between the rate of virtual topology updates and convergence. These insights give us the confidence to claim that a much larger system on the scale of hundreds of thousands of highly heterogeneous nodes would perform as well as our current implementation.
Throughput of wireless networks of untuned radios

The drive toward the implementation and massive deployment of wireless sensor networks calls for ultra-low-cost and low-power nodes. While the digital subsystems of the nodes are still following Moore's Law, there is no such trend regarding the performance of analog components. This work proposes a fully integrated architecture of both digital and analog components (including local oscillator) that offers significant reduction in cost, size and power consumption of the overall node. While such a radical architecture cannot offer the reliable tuning of standard designs, it is shown that by using randomized signal processing techniques, a dense network of such nodes can achieve throughput linear in the number of channels available for communication. Moreover, the ratio of the achievable throughput of the untuned network to the throughput of a tuned network with perfect coordination is shown to be close to 1/e.
High-Speed Action Recognition and Localization in Compressed Domain Videos

We present a compressed domain scheme that is able to recognize and localize actions at high speeds. The recognition problem is posed as performing an action video query on a test video sequence. Our method is based on computing motion similarity using compressed domain features which can be extracted with low complexity. We introduce a novel motion correlation measure that takes into account differences in motion directions and magnitudes. Our method is appearance-invariant, requires no prior segmentation, alignment or stabilization, and is able to localize actions in both space and time. We evaluated our method on a benchmark action video database consisting of six actions performed by 25 people under three different scenarios. Our proposed method achieved a classification accuracy of 90%, comparing favorably with existing methods in action classification accuracy, and is able to localize a template video of 80 x 64 pixels with 23 frames in a test video of 368 x 184 pixels with 835 frames in just 11 s, easily outperforming other methods in localization speed. We also perform a systematic investigation of the effects of various encoding options on our proposed approach. In particular, we present results on the compression-classification tradeoff, which would provide valuable insight into jointly designing a system that performs video encoding at the camera front-end and action classification at the processing back-end.
When Do Redundant Requests Reduce Latency?

Many systems possess the flexibility to serve requests in more than one way, such as distributed storage systems that store multiple copies of the data. In such systems, the latency of serving the requests may potentially be reduced by sending redundant requests : a request may be sent to more servers than needed and deemed served when the requisite number of servers complete service. Such a mechanism trades off the possibility of faster execution of the request with the increase in the load on the system. Several recent works empirically evaluate the latency performance of redundant requests in diverse settings. In this paper, we perform an analytical study of the latency performance of redundant requests, with the primary goals of characterizing under what scenarios sending redundant requests will help (and under what scenarios it will not), and of designing optimal redundant-requesting policies. We show that when service times are i.i.d. memoryless or “heavier,” and when the additional copies of already-completed jobs can be removed instantly, maximally scheduling redundant requests achieves the optimal average latency. On the other hand, when service times are i.i.d. “lighter” or when service times are memoryless and removal of jobs is not instantaneous, then not having any redundancy in the requests is optimal under high loads. Our results are applicable to arbitrary arrival processes.
Morphological motion field representation for region-based image sequence coding

We introduce a novel region-based video compression framework that uses a morphological operation to efficiently capture motion correspondences between consecutive frames in an image sequence. Our coder is built on the observation that the motion field associated with typical image sequences can be segmented into component motion subfield "clusters" associated with distinct objects or regions in the scene, and further that these clusters can be efficiently captured using morphological operators in a "backward" framework that avoids needing to explicitly send object boundaries. Cluster matching is used in lieu of the conventional block matching methods of standard video codecs to define a cluster motion representation paradigm. Experimental coding results show about 10-20% reduction in prediction error energy and 0.3-1 db (average of about 0.44 dB) reduction in the final residue-coded peak signal to noise ratio (PSNR) using our proposed motion compensation framework on the football sequence over standard block motion compensation methods like MPEG, while also requiring less computational complexity.
Some fundamental properties of MMSE filter banks

We design filter banks that are best matched to the input signal statistics in M-channel subband coders, using a broad class of rate-distortion criteria. We present fundamental properties and analytical expressions for minimum mean-squared error (MMSE) filter banks, without constraints on filter length, under optimal bit allocation requirements. We also investigate a constrained-length version of this problem, which is applicable to practical coding scenarios. While the optimal filter banks are nearly perfect-reconstruction at high rates, we show that MMSE FIR filter banks enjoy a significant advantage (in the MSE sense) over optimal perfect-reconstruction FIR filter banks at all rates.
On Secure Distributed Source Coding

Exploiting correlated observations at multiple remote nodes for generating a secret common randomness to facilitate secure communication of data between the terminals has been addressed by many authors [1][2][3]. But in many applications such as sensor networks, the correlated observations might themselves be the data that need to be communicated. We address this problem of secure distributed source coding where we want to leak the least amount of information about the sources to an eavesdropper who may have access to a correlated observation of its own and who can listen in on the communication between the distributed encoders. The focus here is on minimizing the amount of information revealed to the eavesdropper rather than on minimizing the rate required to effect the communication.
Optimal segmentation of a VBR source for its parallel transmission over multiple ATM connections

Variable bit rate (VBR) transmission is widely regarded as the best solution for the transport of compressed image and video data, both in terms of network utilization and quality of data decoded at the receiver. However, significant problems remain unsolved to make this a viable approach. One of these problems is that of efficiently matching characteristics of the VBR source to those of the channel, in order to maximize end-to-end system performance. In this work, we propose a model for the channel based on which the source can make optimal decisions regarding bit allocation and rate control. This model consists of N queues, for each of which the source negotiates with the network statistical performance guarantees, consisting of allowable average transmission and packet loss rates. Based on such a channel description, the source determines how to allocate packets to each queue, to minimize the expected distortion of the images reconstructed at the receiver. A provably optimal algorithm for computing such bit allocations is the core of this work. Simulation results are presented.
A Piggybacking Design Framework for Read-and Download-Efficient Distributed Storage Codes

Erasure codes are being extensively deployed in distributed storage systems instead of replication to achieve fault tolerance in a storage efficient manner. While traditional erasure codes are storage efficient, they can result in a significant increase in the amount of data access and downloaded during rebuilding of failed or otherwise unavailable nodes. In this paper, we present a new framework, which we call piggybacking, for constructing distributed storage codes that are efficient in the amount of data read and downloaded during rebuilding, while meeting requirements arising out of system considerations in data centers—maximum-distance-separability (MDS), high-rate, and a small number of so-called substripes. Under this setting, to the best of our knowledge, piggyback codes achieve the minimum average amount of data access and downloaded during rebuilding among all existing explicit solutions. The piggybacking framework also offers a rich design space for constructing codes for a variety of other settings. In particular, we construct codes that require minimum amount of data access and downloaded for rebuilding among all existing solutions for: 1) binary MDS array codes with more than two parities and 2) MDS codes with the smallest locality during rebuilding. In addition, we show how piggybacking can be employed to enable efficient repair of parity nodes in codes that address the rebuilding of only systematic nodes. The basic idea behind the piggybacking framework is to take multiple instances of existing codes and add carefully designed functions of the data from one instance to the others. This framework provides 25% to 50% savings in the average amount of data access and downloaded during rebuilding depending on the choice of the code parameters.
Frequency-shift-invariant orthonormal wavelet packet representations

It is commonly known that wavelet expansions are very sensitive to translation of an input signal due to their dyadic structure. Without proper time alignment of the signal to the decomposition basis vectors, a compact expansion may be unattainable. Recently, work has been done to combat translation sensitivity by determining the best translation-invariant expansion of a signal with respect to a set of bases. However, the dyadic structure of wavelet expansions also results in frequency alignment sensitivity. In this work, we introduce a frequency-shifted wavelet packet library as well as an efficient best basis algorithm for determining the best signal representation among all frequency shifts of a signal.
Distributed multimedia transmission from multiple servers

We address the problem of streaming video efficiently from multiple servers in a distributed manner. Invoking a scalable video encoding format, we describe a network-friendly, rate-distortion efficient distributed streaming algorithm based on a robust distributed encoding paradigm. The algorithm is efficiently matched dynamically to both the video content and the network channel conditions. Due to the diversity effect of multiple servers, our distributed algorithm is immune to single points of failure, provides natural load-balancing of the multiple servers and a graceful degradation in quality with an increase in multi-server load or a decrease in network throughput.
Distributed code constructions for the entire Slepian-Wolf rate region for arbitrarily correlated sources

Slepian-Wolf coding tackles the problem of distributed encoding of correlated discrete-alphabet sources for decoding at a common receiver. In this work, we propose a distributed linear block code construction for attaining any point on the Slepian-Wolf achievable rate region for arbitrarily correlated sources using only a single code. Specifically, our prescription allows for any arbitrary memoryless joint probability distribution over any arbitrary number of distributed sources, and allows for any arbitrary rate combination that lies in the Slepian-Wolf achievable region. Special cases of our framework include the single source case (wherein our construction reduces to an entropy coder), source coding with side-information at the receiver (so-called corner points of the Slepian-Wolf region), and specific source correlation models (such as induced by a virtual Binary Symmetric Channel model). In this work, we describe how to use low density parity check (LDPC) codes in the proposed framework to solve the general Slepian-Wolf problem constructively.
An integrated source transcoding and congestion control paradigm for video streaming in the Internet

In this work we present an end-to-end optimized video streaming system comprising of synergistic interaction between a source packetization strategy and an efficient and responsive, TCP-friendly congestion control protocol [Linear Increase Multiplicative Decrease with History (LIMD/H)]. The proposed source packetization scheme transforms a scalable/layered video bitstream so as to provide graceful resilience to network packet drops. The congestion control mechanism provides low variation in transmission rate in steady state and at the same time is reactive and provably TCP-friendly. While the two constituent algorithms identified above are novel in their own right, a key aspect of this work is the integration of these algorithms in a simple yet effective framework. This "application-transport" layer interaction approach is used to maximize the expected delivered video quality at the receiver. The integrated framework allows our system to gracefully tolerate and quickly react to sudden changes in the available connection capacity due to the onset of congestion, as verified in our simulations.
Optimized embedded multicarrier modulation for efficient delivery of layered video data

We tackle the problem of efficient image transmission over multicarrier modulation (MCM) systems, proposing the use of a layered or multiresolution (MR) framework. We treat the source as being characterized by multiple layers of importance, and therefore deserving of multiple levels of noise immunity, i.e, having different BER requirements. We present the idea of embedded multicarrier modulation (EMCM) as a very effective way of achieving this, and introduce a fast table-lookup based power allocation algorithm that optimizes the multicarrier constellation design in terms of maximizing the deliverable throughput bit rates for the different resolution layers, subject to a total power constraint. Simulation results of our EMCM system reveal substantial gains (up to about 25%) in deliverable bit rates over optimized TDM-based MCM designs. Further, in typical image transmission simulations using an embedded wavelet image coder, the EMCM approach yields almost 3 dB gains in delivered quality over conventional single-resolution MCM systems.
Multiple-description wavelet based image coding

We consider the problem of image coding for communication systems that use diversity to overcome channel impairments. We focus on the special case in which there are two channels of equal capacity between a transmitter and a receiver. Our designs are based on a combination of techniques successfully applied to the construction of some of the most efficient wavelet based image coding algorithms, with multiple description scalar quantizers (MDSQs). For a given image, we produce two bitstreams, to be transmitted over each channel. Should one of the channels fail, each individual description guarantees a minimum image quality specified by the user. However, if both descriptions arrive at destination, they are combined to produce a higher quality image than that achievable based on individual descriptions. We formulate a discrete optimization problem, whose solution gives parameters of the proposed encoder yielding optimal performance in an operational sense. Simulation results are presented.
Theory of rate-distortion-optimal, constrained filterbanks-application to IIR and FIR biorthogonal designs

We design filterbanks that are best matched to input signal statistics in M-channel subband coders, using a rate-distortion criterion. Previous research has shown that unconstrained-length, paraunitary filterbanks optimized under various energy compaction criteria are principal-component filterbanks that satisfy two fundamental properties: total decorrelation and spectral majorization. In this paper, we first demonstrate that the two properties above are not specific to the paraunitary case but are satisfied for a much broader class of design constraints. Our results apply to a broad class of rate-distortion criteria, including the conventional coding gain criterion as a special case. A consequence of these properties is that optimal perfect-reconstruction (PR) filterbanks take the form of the cascade of principal-component filterbanks and a bank of pre- and post-conditioning filters. The proof uses variational techniques and is applicable to a variety of constrained design problems. In the second part of this paper, we apply the theory above to practical filterbank design problems. We give analytical expressions for optimal IIR biorthogonal filterbanks; our analysis validates a conjecture by several researchers. We then derive the asymptotic limit of optimal FIR biorthogonal filterbanks as filter length tends to infinity. The performance loss due to FIR constraints is quantified theoretically and experimentally. The optimal filters are quite different from traditional filters. Finally, a sensitivity analysis is presented.
Adaptive sleep discipline for energy conservation and robustness in dense sensor networks

This paper presents an adaptive approach for conserving energy in high-density sensor networks. The proposed method allows sensor nodes to sleep while ensuring that application performance constraints are met. Unlike deterministic algorithms that assume static connectivity, the approach uses a randomized algorithm to provide robustness to the variations in network connectivity. These variations are due to fading channels, depletion or addition of nodes, node mobility, and the sleeping of nodes. The algorithm developed in the paper is extremely lightweight and does not require nodes to keep any state information about their individual neighbors. Based on local observations, each node independently decides when to sleep and wakeup. The algorithm also ensures an evenly distributed workload among nodes, and achieves energy savings proportional to the density of nodes.
Robust turbo-based data hiding for image and video sources

Channel coding with side information (CCSI) provides a powerful framework for constructing data embedding codes. As a result, there has been a considerable amount of work done in trying to apply these CCSI codes to the field of image and video watermarking. The CCSI codes that exist in the literature (see Chou, J. et al., 1999; Kesal, M. et al., 2000; Eggers, J. et al., 2001), however, are relatively far from theoretical bounds. Furthermore, these codes, by themselves, are not well-suited for dealing with geometrical distortions which often appear in watermarking attacks. We propose a CCSI code that is closer to the theoretical bounds (within 2.0 dB) than the codes that exist in the literature. In addition, we show how this code construction can be combined with a synchronization code to deal with geometrical distortions.
Recovering K-sparse N-length vectors in O(K log N) time: Compressed sensing using sparse-graph codes

We study the design of measurement matrices for compressed sensing, where the goal is to stably acquire and reconstruct arbitrary K-sparse N-length signals in the presence of noise. We propose a new design framework that simultaneously leads to low measurement cost and low computational cost. In particular, the proposed framework guarantees successful recovery with high probability using O(K log N) measurements with a computational complexity of O(K log N). Both the measurement cost and algorithm runtime are order-optimal for support recovery when K = O (Nδ ) for some 0 <; δ <; 1. To the best of our knowledge, this is the first result that achieves this optimal scaling. The remarkable gains are brought by the proposed measurement structure based on sparse-graph codes, which allows for reconstructions of sparse signals using a simple peeling decoder. More generally, we formally connect general sparse recovery problems with sparse-graph decoding, and demonstrate our design in terms of the measurement cost, computational complexity and performance.
Optimization of high-energy-compaction, nearly-orthonormal, linear-phase filter banks

In this paper, we design two-channel, perfect-reconstruction, linear-phase, biorthogonal filter banks that maximize orthonormality subject to structural constraints on the filter bank. These structural constraints may be adjusted so as to provide various degrees of energy compaction. The applications targeted are in compression. Proper cost functions are formulated and an efficient signal-adaptive optimization algorithm is proposed. Our algorithm is motivated by a number of interesting properties of the correlation matrix of typical signals and images, and efficiently uses the degrees of freedom in the parameterization of perfect reconstruction filter banks. The algorithm provides different levels of tradeoff between the energy compaction and approximate orthonormality goals. Comparisons with the popular Daubechies 9-7 filter bank show significant improvements in terms of orthogonality and energy compaction.