# Inter-Data Center RDMA: Challenges, Status, and Future Directions
## 3 Inter-DC RDMA Requirements and Challenges
These algorithms are designed for low-latency environments, which adjust the sending rate according to ECN, RTT, or other timely feedback, so that they struggle to adapt to the high-latency conditions of inter-DC networks. The long RTT slows down the convergence of these algorithms, leading to oscillating sending rates and queue lengths, which can impair throughput and cause queue starvation. **Buffer pressure:** Lossless RDMA requires at least one bandwidth-delay product (BDP) buffer space to avoid packet loss [19], and the switches in DCs usually have shallow buffers in order to ensure low queuing delay. However, the latency across DCs far exceeds that within a DC, and the inter-DC link capacities are usually very large, resulting in large buffer pressure on the egress switches connected to long-haul dedicated links. Therefore, if intra-DC RDMA solutions are simply deployed in inter-DC scenarios with traditional switches, this can lead to buffer overflow and packet loss, which further complicates the retransmission process. Consequently, the packet loss issues caused by buffer pressure not only pose challenges to loss packet recovery mechanisms, resulting in higher latency and lower bandwidth utilization, but also impose a stricter requirement on the design of switches for inter-DC interconnection scenarios. **Loss packet recovery:** When RDMA solutions that rely on lossless networks are deployed across DCs, RDMA's congestion control mechanism may struggle to respond promptly to network state changes. RDMA typically uses a GBN retransmission mechanism, where the sender retransmits all packets starting from the lost one. In high-latency environments, this can result in substantial retransmission overhead, reducing transmission efficiency. Therefore, the sluggishly responsive congestion control not only influences the detection of packet loss but also leads to higher packet loss rates and triggers more frequent retransmissions, exacerbating network congestion. For lossy RDMA solutions, the impact of inter-DC deployment on packet loss recovery is insignificant as well. Lossy solutions permit switches to drop packets, with the sender retransmitting lost packets when specific events are triggered, called selective retransmission. Typical examples of lossy solutions include IRN [28] for RoCEv2 and traditional TCP. Unlike PFC, IRN sends all packets within a sliding window (i.e., a bitmap), and packets are dropped by switches when buffer overflow occurs. The fixed-size sliding window limits the number of inflight packets and affects data transmission performance. Selective retransmission can achieve good performance with short RTTs (i.e., DCN environment), and inter-DC long-haul transmission introduces extremely larger RTTs than intra-DC links. IRN's static bitmap is too rigid to balance the intra-DC and inter-DC traffic simultaneously. Furthermore, it is difficult for the sender to perceive packet loss in time when congestion occurs at downstream DCs, and then packet retransmission also introduces high flow latency due to the large inter-DC RTT. Therefore, current lossy solutions cannot meet the requirements of high throughput and low latency for inter-DC applications. **Load balance:** In DCNs, multiple equal-cost parallel paths are available between end-host pairs. Traditional RDMA uses single-path transmission, failing to fully utilize the abundant network capacity. To leverage multipath capabilities, researchers have proposed various multipath transmission and load-balancing mechanisms for intra-DC RDMA solutions. These solutions (e.g., MP-RDMA [38], PLB [39], and Proteus [40]) allow for multiple paths to balance total bandwidth, enhancing throughput and minimizing the flow completion time (FCT) by fully utilizing parallel paths. The multipath-enabled load-balancing process generally consists of two stages. First, the congestion status of candidate multipaths is obtained, and initial traffic distribution considers these statuses to evenly allocate data flows across paths.