# Experimenting with Routing Protocols in the Data Center: An ns-3 Simulation Approach
###### Abstract  
Massive scale data centers (MSDC) have become a key component of current content-centric Internet architecture. With scales of up to hundreds of thousands servers, conveying traffic inside these infrastructures requires much greater connectivity resources than traditional broadband Internet transit networks. MSDCs use Fat-Tree type topologies, which ensure multipath connectivity and constant bisection bandwidth between servers. To properly use the potential advantages of these topologies, specific routing protocols are needed, with multipath support and low control messaging load. These infrastructures are enormously expensive, and therefore it is not possible to use them to experiment with new protocols; that is why scalable and realistic emulation/simulation environments are needed. Based on previous experiences, in this paper we present extensions to the ns-3 network simulator that allow executing the Free Range Routing (FRR) protocol suite, which support some of the specific MSDC routing protocols. Focused on the Border Gateway Protocol (BGP), we run a comprehensive set of control plane experiments over Fat-Tree topologies, achieving competitive scalability running on a single-host environment, which demonstrates that the modified ns-3 simulator can be effectively used for experimenting in the MSDC. Moreover, the validation was complemented with a theoretical analysis of BGP behavior over selected scenarios. The whole project is available to the community and fully reproducible. ns-3; routing; data center +
Footnote †: journal:  
definition  
networks, originally conceived to build non-blocking switching matrices for telephone networks, are made up of multiple levels of switches, where each switch of one level is connected to all those of the next level, obtaining a high path diversity as a result. In a previous work [3], we experimented with data center routing protocols in emulated environments such as Kathara [4; 5], Megalos [6], CORE [7] or Mininet [8], complementing the work presented in [9], where the Sibyl framework is used for evaluating implementation of routing protocols in fat-trees, including the Border Gateway Protocol (BGP) in the data center [10], Openfabric (IS-IS with flooding reduction) [11], and Routing in Fat Trees (RIFT) [12; 13]. This framework presents wall-clock independent metrics, which permits us to normalize the results disregarding the underlying execution environment. These previous works are based on the routing protocols from the Free Range Routing (FRR) suite, an open source implementation of BGP, OSPF, RIP, IS-IS, and other protocols, inheriting the code base of the Quagga project [14]. As mentioned in the previous work, emulated devices run exactly the same firmware of hardware devices, therefore implementing identical functionality. Moreover, emulated devices are exposed to real-life software errors, which permits us to not only evaluate functionality, but also resilience. On the other hand, re-implementation of network protocols and applications is needed for discrete event simulation, weakening the chances of testing real use cases. Nevertheless, a simulator provides an environment for replicable experiments always guaranteeing the same conditions and provides fine management of the timing issues. With these considerations in mind, in this paper we present a port of FRR, to the Direct Code Execution (DCE) [15] mode of the ns-3 Network Simulator [16]. ns-3 is a discrete event network simulator for Internet systems, widely supported in the networking community. ns-3 has a mode of execution called DCE [17], which allows using native code (properly compiled) in the simulations. In this way, it is possible to execute existing implementations of network protocols or applications within ns-3. Therefore, it is possible to reconcile the virtues of discrete event simulators with emulation, which preserves the real implementation of protocols and applications.