Getting Started with Azure Load Testing
Tags: performance, azure, testing, webperf
Published: 2021-12-03 21:33:13

<!-- wp:paragraph -->
<p>A new sheriff in town. Microsoft launched the fully managed Azure service for performance test engineers and developers to perform high scale load testing called Azure Load Testing, currently in preview mode. It is quite a big announcement for performance engineers. Let us see whether Azure Load Testing fulfills our needs or not in this blog post.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>What is Azure Load Testing?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>As foretold, aptly put, Azure Load Testing is an offering for the customers who are in the Azure ecosystem. Even if you are not in Azure, you can still spin up the load test resource for performance testing. But the catch is, PRICING. We can discuss the pricing in another blog post. Let us focus on the features, how to get started and more.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Azure Load Testing is backed up by none other than Apache JMeter and Azure infrastructure. Here is a meme I created after the announcement was made.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8901,"sizeSlug":"full","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-full"><a href="https://qainsights.com/wp-content/uploads/2021/12/Azure-JMeter.jpg"><img src="https://qainsights.com/wp-content/uploads/2021/12/Azure-JMeter.jpg" alt="Azure Load Testing Meme" class="wp-image-8901"/></a><figcaption>Azure Load Testing Meme</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:generateblocks/headline {"uniqueId":"2cf8a61b"} -->
<h2 class="gb-headline gb-headline-2cf8a61b gb-headline-text">Azure Services</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>In Azure Load Testing, the core component is the latest version of Apache JMeter 5.4.1 and the Open JDK 64-bit 15.0.5. Microsoft has its own flavor of <a href="https://docs.microsoft.com/en-us/java/openjdk/download" target="_blank" rel="noreferrer noopener">JDK</a>. But I am not sure whether Azure Load Testing uses Microsoft distribution or other flavors. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Tests will be executed on Linux AMD instance with 4 cores and 16&nbsp;GB RAM.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Apart from JMeter and JDK, following are the dependencies which will be created for performance testing:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>App Service</li><li>Kubernetes</li><li>Application Insights</li><li>Virtual Machine</li><li>Azure SQL Server</li><li>Storage account</li><li>Azure Monitor (for monitoring)</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Here is the sample dashboard:</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8902,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/QAInsights-12-2-2021-10.39.46-PM-000000.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/QAInsights-12-2-2021-10.39.46-PM-000000-1024x450.png" alt="Azure Monitor" class="wp-image-8902"/></a><figcaption>Azure Monitor</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:generateblocks/headline {"uniqueId":"1b388c80"} -->
<h2 class="gb-headline gb-headline-1b388c80 gb-headline-text">Getting Started with Azure Load Testing</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>To get started with Azure Load Testing, the following are the prerequisites:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Azure account with active subscription</li><li>JMeter script and its dependencies</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Creating an <a href="https://azure.microsoft.com/en-us/free/" target="_blank" rel="noreferrer noopener">Azure account</a> is simple and free. If you are new to Azure, you will get a $200 credit for 30 days.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Assigning roles</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Before creating Azure Load Testing resources, let us create a Management Group and assign a valid role to spin up the resource.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>In the Azure portal, search for <strong>Management Groups</strong>. In <strong>Management Groups</strong>, click on <strong>Create </strong>and enter the valid <strong>Management group ID</strong> and then click on <strong>Submit</strong>. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Management Groups</strong> help to organize the various entities such as policies, compliance and more.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8904,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-1024x644.png" alt="Management Groups" class="wp-image-8904"/></a><figcaption>Management Groups</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Inside the created management group, it is important to add an active subscription. Click on <strong>Add Subcription</strong> and select the <strong>active subscription</strong> and then click on <strong>Save</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8906,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-2.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-2-1024x516.png" alt="Active Subscription" class="wp-image-8906"/></a><figcaption>Active Subscription</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>To spin up the resources, it is critical to have valid roles assigned. To assign a <strong>Load Test Owner</strong> role, click on <strong>Access Control (IAM) </strong>and then click on the <strong>Add role assignment</strong> button as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8907,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-3.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-3-1024x446.png" alt="Add a Role" class="wp-image-8907"/></a><figcaption>Add a Role</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>In the <strong>Add role assignment</strong> window, search for "<em>load</em>" and then select the appropriate role. In this example, I selected <strong>Load Test Owner</strong>. Click on the <strong>Next </strong>button to add the members.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8909,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-5.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-5-1024x570.png" alt="Role Assignment" class="wp-image-8909"/></a><figcaption>Role Assignment</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Click on <strong>Select Members</strong> and then add the valid members. Click on <strong>Next </strong>to <strong>Review and Assign</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8908,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-4.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-4-1024x503.png" alt="Select Members" class="wp-image-8908"/></a><figcaption>Select Members</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p></p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Create a new Azure Load Testing resource</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>To begin with the performance testing, the first step is to create a new <strong>Azure Load Testing</strong> resource. Go to the home page of Azure portal, and enter <em>Load Test </em>in the search text bar. </p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8910,"sizeSlug":"full","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-full"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-6.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-6.png" alt="Getting Started with Azure Load Testing" class="wp-image-8910"/></a><figcaption>Getting Started with Azure Load Testing</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Select <strong>Azure Load Testing</strong> under <strong>Services</strong> as shown above.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>On the Azure Load Testing page, click on <strong>Create </strong>to create a new Load Test Resource. Under the <strong>Basics </strong>tab, fill in the mandatory fields as shown below. Then, click on the <strong>Review + create</strong> button. </p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8911,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-7.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-7-1024x881.png" alt="Create Load Test Resource" class="wp-image-8911"/></a><figcaption>Create Load Test Resource</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>After validation is successful, click on <strong>Create</strong>. This will initialize the deployment and, once it is successfully completed, you will get the status.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8912,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-8.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-8-1024x386.png" alt="Deployment Status" class="wp-image-8912"/></a><figcaption>Deployment Status</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Click on the <strong>Go to resource</strong> button to view the landing page of Azure Load Testing. Welcome home :) </p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8913,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-9.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-9-1024x370.png" alt="Home page of Azure Load Testing" class="wp-image-8913"/></a><figcaption>Home page of Azure Load Testing</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:generateblocks/headline {"uniqueId":"d64c431b","element":"h3"} -->
<h3 class="gb-headline gb-headline-d64c431b gb-headline-text">Create a new test</h3>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>Click on the <strong>Create </strong>button to start a new test for performance testing. It is a 7 - step easy process. </p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8914,"sizeSlug":"full","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-full"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-10.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-10.png" alt="Create a new test" class="wp-image-8914"/></a><figcaption>Create a new test</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:list {"ordered":true} -->
<ol><li>In the <strong>Basics </strong>tab, enter the test name and description.</li><li>In <strong>Test plan</strong>, upload the JMeter test plan and its dependencies. Only one test plan is allowed. </li><li>In <strong>Parameters</strong>, enter sensitive information like API keys, Passwords and more.</li><li>In <strong>Load</strong>, configure the number of engines you need for your test. 1 engine will spin up 250 threads.</li><li>In <strong>Test criteria</strong>, configure the SLAs</li><li>In <strong>Monitoring</strong>, you can link the resources to monitor</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Click on <strong>Review + create</strong> and then click on <strong>Create</strong>, which will spin up the test.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>After the validation success, the test will start. A few moments later, you will see the dashboard as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8917,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-12.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-12-1024x517.png" alt="Runtime Dashboard" class="wp-image-8917"/></a><figcaption>Runtime Dashboard</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>During the test, you can stop the test, refresh the dashboard, share the dashboard, and download the input file.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>After the test is done, you can rerun the test, compare the test, download the logs and results and delete the run.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Delete the Azure Load Test resource</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Once the test is done, please make sure that you are deleting the resource. Before deleting it, download all the artifacts.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To delete, go to the <strong>Azure Load Test</strong> resource, click on <strong>Delete</strong>, type <strong>yes</strong>, select the reason and then click on the <strong>Delete </strong>button as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8919,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/12/image-14.png"><img src="https://qainsights.com/wp-content/uploads/2021/12/image-14-1024x503.png" alt="Delete Azure Load Testing resource" class="wp-image-8919"/></a><figcaption>Delete Azure Load Testing resource</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:generateblocks/headline {"uniqueId":"39f1b298"} -->
<h2 class="gb-headline gb-headline-39f1b298 gb-headline-text">Limitations and Cons</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>Every solution will have its own limitations. Azure Load Test comes with the following limitations from my exploration:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>Only one JMeter test plan is allowed to upload</li><li>a limited number of regions.</li><li>A maximum of 25,000 virtual users is supported.</li><li>A maximum of 3 hours duration is supported. If you want to run endurance testing, you cannot trigger.</li><li>Dashboard update is flaky. </li><li>Expensive solution</li></ol>
<!-- /wp:list -->

<!-- wp:heading -->
<h2>Conclusion</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Azure Load Test offering is a great solution if you are into Azure ecosystem and your performance test requirements aren't crazy. The intuitive user interface and fluid workflow makes it easy to trigger the test. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The essential features such as creating tests, dashboards, compare, download logs/results and more have been implemented. But features such as network virtualization, multiple tests support, more virtual users, more duration, split the load across multiple engines or regions, built-in JMeter plugins support, custom snippet support and more are missing.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Give the Azure Load Test a shot, you may like it. </p>
<!-- /wp:paragraph -->