Addressing TCP/IP processing challenges using the IA and IXP processors

The majority of datacenter applications such as web services, e-commerce, storage, and firewall use Transmission Control Protocol/Internet Protocol (TCP/IP) as the data communication protocol of choice. As such, the performance of these applications is largely dependent upon the efficient processing of TCP/IP packets. In addition, with the arrival of the 10 Gigabit Ethernet, the TCP/IP packet processing needs to become faster and more efficient to let the applications benefit from higher bandwidths. Motivated by this, the material presented in this paper focuses on (a) network bandwidth and the associated TCP/IP processing requirements of typical datacenter applications, (b) challenges of scaling TCP/IP packet processing to 10 Gigabit speeds, and (c) exploring to what extent these challenges can be addressed by the Intel IA32 and IXP network processors. We also discuss potential techniques that further help TCP/IP processing. INTRODUCTION TCP/IP over Ethernet is the most dominant packet processing protocol in datacenters and on the Internet. It is used by both server and network infrastructure applications (e.g., firewall, proxy server, etc.). We show that the TCP/IP component of these applications constitutes a significant amount of the overall processing and hence it is important to understand its characteristics and make sure that it can scale well up to 10 Gbps and beyond. Analysis done on TCP/IP in the past [1] has ® Intel is a registered trademark of Intel Corporation or its subsidiaries in the United States and other countries. shown that only a small fraction of the computing cycles are required by the actual TCP/IP processing and that the majority of cycles are spent in dealing with the Operating System (OS), managing the buffers, and passing the data back and forth between the stack and the end-user applications. Many improvements and optimizations have been done to speed up the TCP/IP packet processing over the years. CPU-intensive functions such as checksum calculation and segmentation of large chunks of data into right-sized packets have been offloaded to the Network Interface Card (NIC). Interrupt coalescing by the NIC devices to minimize the number of interrupts sent to the CPU is also reducing the burden on processors. In addition to these NIC features, some OS advancements such as asynchronous IO and pre-posted buffers [12] are also helping to speed up TCP/IP packet processing. While these optimizations, combined with the increases in CPU processing power, are sufficient for processing packets at 100 Mbps and 1 Gbps Ethernet speeds, these are clearly not sufficient for 10 Gbps speeds. Upcoming applications like storage over IP, web services, and ubiquitous broadband connectivity to homes enabling video-on-demand applications require multiGbps speeds that require TCP/IP packet processing to scale to these speeds with the least amount of computing power. Figure 1 shows the transmit and receive-side throughput and CPU utilization data for the Microsoft Windows 2003 Enterprise Edition TCP/IP stack running on an Intel Xeon processor (2.6 GHz) with * Other brands and names are the property of their respective owners. TM Xeon is a trademark of Intel Corporation or its subsidiaries in the United States and other countries. Intel Technology Journal, Volume 7, Issue 4, 2003 Addressing TCP/IP Processing Challenges Using the IA and IXP Processors 40 Hyper-Threading Technology. These measurements were taken using NTttcp, a variant of the ttcp micro benchmark [11] optimized for the Windows OS. This paper is organized as follows. We perform an indepth measurement and evaluation of packet processing on Intel’s state-of-the-art server and network processors. The primary focus of this study is to (a) quantify the computational requirements of the TCP/IP packet processing component within server and network infrastructure workloads, (b) identify and analyze the challenges of scaling the TCP/IP packet processing to 10 Gbps in both the server and network infrastructure environments, and (c) show how Intel server and network processors are addressing some of these challenges by discussing the relevant Intel research. TCP/IP Throughput and CPU Utilization 0 50