experimental study of protocol-independent redundancy elimination algorithms
The idea of identifying and removing repetitive patterns in the network data transfers, also known as protocol-independent redundancy elimination , and its benefits have received thorough consideration. However, actual implementation of such systems received much less attention. The intention of the redundancy elimination is to increase capacity of low-bandwidth network connections, when searching for redundancies and replacing them is faster than transmitting unprocessed redundant data. As long as network is slow, any reasonable implementation is beneficial. But as network capacities grow, the maximal throughput that system can provide becomes critical for its deployment. Thus, an appropriate choice of redundancy eliminating algorithm and its parameters becomes very important. This work addresses the problem of algorithm and parameter selection. We describe possible variations of the basic scheme, and demonstrate experiments that we have conducted for each variation. We discuss the trends observed in the results and explain their nature. We then propose a methodology to make a choice of the algorithm and its parameters, based on the obtained measurements.
redundancy elimination