Testing the Waters of AWS EC2 C5D Instances
Tags: awsec2, nvme, solutionsarchitect, awsfordevops
Published: 2018-11-21 10:20:23

---
title: Testing the Waters of AWS EC2 C5D Instances
published: true
description: Ever since Amazon announced AWS EC2 C5D instances, we -- as AWS practitioners -- have been digging deep into the technicalities of it. We found some interesting stuff, and we’d like to share our findings for the greater good. 
tags: AWS EC2, NVMe, Solutions Architect, AWS for DevOps
---

<span style="color: #000000;">Ever since Amazon announced AWS EC2 C5D instances, we -- as AWS practitioners -- have been digging deep into the technicalities of it. We found some interesting stuff, and we’d like to share our findings for the greater good.</span>
<h2><span style="color: #000000;"><b>C5D Intro: AWS EC2 Instances with Local NVMe Storage</b></span></h2>
<span style="color: #000000;">For those unaware, AWS EC2 C5D instances are high-performance block storage instances. Introduced sometime during May 2018, they are said to be ideal for applications that need access to high-speed, low latency local storage. This has made it media vertical’s favorite instance out of the lot. If you want to know more about its applications and other details, <a href="https://blog.totalcloud.io/testing-the-waters-of-aws-ec2-c5d-instances/">click here.</a>     </span>

<h2><span style="color: #000000;"><b>Tests</b></span></h2>


<span style="color: #000000;">To test the performance of C5D instances, we launched a C5D.large instance in US-east 1 region with 9GB Amazon Elastic Block Store + 50GB Amazon EC2 NVMe Instance Storage. </span>

<span style="color: #000000;">In the same region and availability zone, we launched a C5.large instance with 54 GB EBS volume and T2.small instance for comparison with no NACLs in the way, and open security groups between the instances. All instances were running Amazon Linux 2 OS (ami-0a5e707736615003c), and patched up to date as of October 2018.</span>

<blockquote><strong><span style="color: #000000;">Before we walk you through the benchmarking results, let's have a look at the pricing comparison between C5D.Large, C5.Large and T2.Small on-demand instances:</span></strong></blockquote>


<img class="alignnone wp-image-1012" src="https://thepracticaldev.s3.amazonaws.com/i/z0v6llttckxf5dop2x9d.png" alt="AWS EC2 C5D instance pricing comparison with C5 Large and T2 Small" width="685" height="242" />

<span style="color: #000000;">We ran the Sysbench tool to calculate all prime numbers up to 20,000 to compare and contrast the pure compute performance of C5D instances with C5 instance as well as the the most popular instance type, T2. </span>
<h3><span style="color: #000000;">Here are the kind of tests we ran on all the three instances:</span></h3>
<ul>
 	<li><span style="color: #000000;">To run CPU tests, we made use of the SysBench tool, with a single threaded CPU test.</span></li>
 	<li><span style="color: #000000;">To measure disk performance, we made use of Sysbench file IO benchmark (using random r/w). As these instances are capable of bursting, we conducted few tests to drain the IO balance. Before draining the IO balance, we used IOping to test latency. Then, using FIO, we attempted to drain the balance and test the performance as the burst bucket was emptied and refilled.</span></li>
 	<li><span style="color: #000000;">To benchmark the performance of a typical file server workload, we made use of the blogbench score.</span></li>
 	<li><span style="color: #000000;">To run network tests and demonstrate TCP/IP latency, we used average latency of 100 icmp packets. Additionally, we used iperf to test the bandwidth between devices.</span></li>
 	<li><span style="color: #000000;">To get a more a slightly more "real-world" test, we used the phoronix test suite to run a benchmark compiler test (compiling Linux). This gives real world disk IO across a filesystem alongside general system and CPU performance. Disk IO is often a constraining factor in machine performance, but it very much depends on what particular workload you're running.</span></li>
</ul>
<h3><span style="color: #000000;">Here are the results:</span></h3>
<img class="alignnone wp-image-1009" src="https://thepracticaldev.s3.amazonaws.com/i/amhuve37swjpdghawvlf.png" alt="AWS EC2 C5D Instance Performance Benchmarking with C5L and T2S" width="602" height="255" />
<h3><span style="color: #000000;">Here are Our Key Observations:</span></h3>
<span style="color: #000000;">#1 | The CPUs all come out at the same score, as all three instances have CPUs of the same capability.</span>

<span style="color: #000000;">#2 | The C5D vastly outperforms the C5 and T2 in File IO due to the nvme disk, with nearly 4 times the read and write capability of a C5 instance. AWS uses the same underlying disk technology for C5 and T2. Hence the results for C5 and T2 are similar.</span>

<span style="color: #000000;">#3 | The disk latency measure (IOping) shows that the nvme disk has half the latency of the C5. This is a dramatic difference, and has significant implications for latency-sensitive workloads such as real-time analytics.</span>

<span style="color: #000000;">#4 | The network measures are essentially the same for C5D and C5 instances, as expected, but significantly improved over the T2 instance.</span>

<span style="color: #000000;">#5 | The compile time is interesting. The task hits a CPU bottleneck at this point. Although the C5D is faster every time, it's hard to find a repeatable real-world task that really stretches the capability of the new nvme disk.</span>

<span style="color: #000000;">#6 | The test showed that the CPU maxes out to 100% to compile Linux. This indicates that C5D is a little faster. Even though compiling Linux C5D is a very disk intensive operation, the CPU was the ultimate bottleneck of the task in our test case.  </span>

<img class="alignnone wp-image-1010" src="https://thepracticaldev.s3.amazonaws.com/i/nlc0a3f8havnbbtca3fu.png" alt="AWS EC2 C5D Large instance CPU Maxing out graph" width="592" height="302" />

<span style="color: #000000;">Just to add to these tests, the C5 instance took 8.10 seconds to unpack the Linux kernel. The C5D took 8.06 seconds to unpack.</span>
<h2><span style="color: #000000;"><b>The Wrap UP</b></span></h2>
<span style="color: #000000;">There were several tests conducted over a week to benchmark C5D.Large instance’s performance. Hope these test results gives you the real perspective of the new C5D instances. </span>

<span style="color: #000000;">If you’d like to know more about these test results, write to us at tech@totalcloud.io or tweet to us at <a href="https://twitter.com/totalcloudio">@totalcloudio</a>. Also, do share your views on this post.   We would love to hear your thoughts.
</span>


<span style="color: #000000;"> TotalCloud Inc. has rolled a new  <a href="http://blog.totalcloud.io/reinforce-aws-security-posture-multiple-levels-seconds/">Security Group View</a> and <a href="http://blog.totalcloud.io/gain-full-visibility-aws-network-stack-seconds/"> Network View</a>, which will provide visual cues to network and security loopholes in real-time and in 3D space. <a href="https://totalcloud.io/user-signup.html">Sign-up to try.</a>

</span>

<span style="color: #000000;"> Note: Originally published at <a href="https://blog.totalcloud.io/testing-the-waters-of-aws-ec2-c5d-instances/" target="_blank" rel="noopener"> blog.totalcloud.io </a> on November 9, 2018.
</span>