BENCHMARK RESULTS

NGINX Plus Running with Onload®
Sees a 700% Performance Gain

What is NGINX?
Open source NGINX is an HTTP and reverse proxy server, and a generic TCP/UDP proxy server. NGINX Plus is a software
load balancer, web server and content cache built on top of open source NGINX. NGINX Plus has exclusive enterprisegrade features beyond what’s available in the open source offering, including session persistence, configuration via
API, and active health checks.
NGINX Plus is heavily network dependent by design, so its performance can be significantly improved throughenhancements to the underlying networking layer. NGINX has been used extensively to host some of the biggest
websites including Dropbox, Netflix and Wordpress.com.

Key Observations from Performance Testing• Solarflare’s Onload delivers a 698% performancegain in connections per second for NGINX Plus as a
web server when using 25GbE with 28 NGINX Plusworkers. With 100GbE the connection per second
gains top out at 676%.
• When processing requests per second using 28 orfewer NGINX Plus workers using Onload on 25GbEwith both 0KB and 1KB payloads, it consistently
delivers between 102-258%. At 100GbE the benefit
range is 70-202%.

• We can see from the graph above that the kernel
reaches an asymptote at 290K connections/second while Onload enables scaling to 2,334K
connections/second. In this case both the balanced
and performance profiles for Onload performed thesame.
• When we review the requests/second graph below
at 100GbE we can see that for 0KB payloads Onload
achieves a performance benefit of 126-202%. As wemove up to 1KB payloads the gain is 70-155%.

BENCHMARK RESULTS

Why NGINX Plus Benefits from Kernel Bypass
Since NGINX Plus is network intensive, every connection and every request includes network processing overhead.
Whenever an application like NGINX Plus touches hardware, other than the CPU or memory, and in this case thenetwork, it must make at least one, and sometimes several calls to the operating system kernel. Each request isadditional overhead that requires both CPU cycles and processing time. Solarflare’s Onload moves the network
processing required by NGINX Plus from the kernel into NGINX Plus’s own application space in memory. This single
modification improves NGINX Plus performance by anywhere from 70% to as much as 698% as seen in these graphs.

Description of Test Platforms
For this testing we used two dual socket Intel
Xeon systems labeled “A” and “B”, and three
production networks, 10GbE, 25GbE and100GbE, leveraging a single switch dedicatedto this testing, no other traffic exists on thisswitch. The “A” system was used as the server
and it had two Platinum 8153 CPUs clocked
at 2GHz with 16 cores per processor, the “B”
system was used as the client utilizing twoGold 5120 CPUs clocked at 2.2Ghz but with
14 core processors. Both systems contain
192GB of memory, they boot from an NVMe
flash drive, and have three Solarflare network
cards: SFN8522 dual port 10GbE, X2522-25G
dual port 25GbE and an X2541 single port
100GbE. This enables us to test performanceagainst a range of shipping Solarflare adapters
as shown above.

wrk - a HTTP
benchmark tool

1 GbE

sfocr740a
2x Intel® Xeon®
Platinum 8153 CPU(16c) @2.00GHz
192GB @2666MHz

sfocr740a
2x Intel® Xeon®
Platinum 8153 CPU(16c) @2.00GHz
192GB @2666MHz

10 GbE 25 GbE 100 GbE$5148F-ON

BENCHMARK RESULTS

Tuning Configuration
Here are the changes we made to the standard install
beyond simply leveraging Onload.
• We increased the port range to use 9000 to 65000• Set huge pages equal to 10000• We increased the maximum number of open files
from 1,024 to 8,388,608• Increased the number of files a process can open to8,388,608.
Observations
As a web server, the performance of NGINX Plus canbe gated by both networking and storage performance.
For these tests everything was from memory so we
could more accurately study the benefit that Onload
could provide to this application. NGINX Plus relieson the operating system’s communications stack
to process network I/O requests, but in high core
count environments, this stack has become the newbottleneck. Here are some additional points to consider:• NGINX Plus with 28 workers and Onload can service
up to 4.5 million requests/second, while NGINX Plususing the operating system kernel can only handle
1.7 million requests/second, a 170% improvement.
• Therefore, NGINX Plus with Onload can potentiallyreduce your capex by 62%. In simple terms every two
NGINX Plus servers leveraging Onload can service
the same number of requests as five NGINX Plus
servers using the standard Linux kernel.
• Conversely, adding Onload and a Solarflare 25GbE or
100GbE NIC to existing servers will provide additional
headroom for growth or unanticipated businesspeaks as shown above.
For More Testing Details
Check out Solarflare’s Onload NGINX Web Server
Cookbook for the exact installation and testing process
along with the specific tuning and tweaking commands
executed above.

The above benefit statements are the result of benchmarking designed to focus on the value of optimizing networking through Onload kernel
bypass. Real world use cases are not the same as benchmarks and as such the role that networking plays may vary, so your overall measurable
benefits may be different.

For more information please visit:
solarflare.com

Contact Us:
US +1 949 581 6830
UK +44 (0) 1223 477171
HK +852 2624 8868
Email: sales@solarflare.com

SF-122197-CD Issue 4
NGINX Plus Benchmark Results 112219Copyright © 2019 Xilinx, Inc. Onload and all other designated brands included herein are trademarks of Xilinx in the United States and othercountries. All other trademarks are property of their respective owners.