Cluster-to-cluster data transfer with data compression over wide-area networks ?
We model all the system components involved in end-to-end data transfer as a graph. We formulate the problem to optimize data transfer throughput using parallel flows. We propose a novel software I/O stack with variable parallel data flows per layer. Optimized parallel data flows improve overall data transfer throughput. The novel I/O stack with data compression alleviates network bottleneck situations.
optimization disk-to-disk data transfer system modeling high-speed networks