Abstract-High-performance computing systems are shifting away from traditional interconnect topologies to exploit new technologies and to reduce interconnect power consumption. The Dragonfly topology is one promising candidate for new systems, with several variations already in production. It is hierarchical, with local links forming groups and global links joining the groups. At each level, the interconnect is a clique, with a link between each pair of switches in a group and a link between each pair of groups. This paper shows that the intergroup links can be made in meaningfully different ways. We evaluate three previouslyproposed approaches for link organization (called global link arrangements) in two ways. First, we use bisection bandwidth, an important and commonly-used measure of the potential for communication bottlenecks. We show that the global link arrangements often give bisection bandwidths differing by 10s of percent, with the specific separation varying based on the relative bandwidths of local and global links. For the link bandwidths used in a current Dragonfly implementation, it is 33%. Second, we show that the choice of global link arrangement can greatly impact the regularity of task mappings for nearest neighbor stencil communication patterns, an important pattern in scientific applications.
Hastings et al. conduct a theoretical analysis on global link arrangements and show that the commonly-used absolute link arrangement leads to a smaller bisection bandwidth when the ratio of global/local link bandwidths is larger than 1.25 REF .
Comparing Global Link Arrangements for Dragonfly Networks
2015 IEEE International Conference on Cluster Computing
2015 IEEE International Conference on Cluster Computing
Computer Science