Dynamic core affinity for high-performance file upload on Hadoop Distributed File System
We analyze the impact of core affinity on both network and disk I/O performance. Both parallelism and locality are important for tasks that access disk and network. We suggest a novel approach to dynamically decide the core affinity of HDFS threads. Our dynamic core affinity improves the file upload throughput more than 30%.
affinity hadoop distributed file system big-data multi-core process scheduling