Abstract-This paper presents an evaluation and comparison of three topologies that are popular for building interconnection networks in large-scale supercomputers: torus, fat-tree, and dragonfly. To perform this evaluation, we propose a comprehensive methodology and present a scalable packet-level network simulator, TraceR. Our methodology includes design of prototype systems that are being evaluated, use of proxy applications to determine computation and communication load, simulating individual proxy applications and multi-job workloads, and computing aggregated performance metrics. Using the proposed methodology, prototype systems based on torus, fat-tree, and dragonfly networks with up to 730K endpoints (MPI processes) executed on 46K nodes are compared in the context of multijob workloads from capability and capacity systems. For the 180 Petaflop/s prototype systems simulated in this paper, we show that different topologies are superior in different scenarios, i.e. there is no single best topology, and the characteristics of parallel workloads determine the optimal choice.
In REF , the authors perform a comparison of the torus, fat-tree and dragonfly network topologies by using the TraceR-CODES packet-level simulation framework on network sizes with up to 730K endpoints.
Evaluating HPC Networks via Simulation of Parallel Workloads
SC16: International Conference for High Performance Computing, Networking, Storage and Analysis
SC16: International Conference for High Performance Computing, Networking, Storage and Analysis
Computer Science