Distributed Load Testing in JMeter using Raspberry Pi 400
Tags: performance, testing, webperf, tutorial
Published: 2020-12-23 18:22:49

<!-- wp:paragraph -->
<p>Recently I got the latest version of Raspberry Pi which is 400. I got it as a complimentary from <a href="https://www.zebrium.com/" target="_blank" rel="noreferrer noopener nofollow">Zebrium</a> during Linux Foundation KubeCon 2020. I have to put it for good use. I already implemented <a href="https://qainsights.com/how-to-deploy-pi-hole-in-your-home-network/" target="_blank" rel="noreferrer noopener">Pihole using Raspberry Pi</a>. Later I destroyed the Pihole and started using <a href="https://qain.si/nextdns" target="_blank" rel="noreferrer noopener">NextDNS</a>. Using Pi 400, I implemented distributed load testing in JMeter. This blog post will focus on distributed load testing in JMeter using Raspberry Pi 400.</p>
<!-- /wp:paragraph -->

<!-- wp:generateblocks/headline {"uniqueId":"736acdbc"} -->
<h2 class="gb-headline gb-headline-736acdbc gb-headline-text">What is Distributed Load Testing?</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>Running your load test in your local machine is not a good idea. You will reach the resource limits quickly and you cannot scale your resources or the load. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>You will also have background processes running which might interfere your testing. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>By distributing the load to multiple nodes, you can easily scale the load and resources (vertical scaling). You can spin up the cloud instances, set up the load test environment and destroy the instances. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If you do not want to burden yourself in setting up and maintaining the instances, you can opt in JMeter-on-Cloud solutions like RedLine13.</p>
<!-- /wp:paragraph -->

<!-- wp:generateblocks/headline {"uniqueId":"996293b5"} -->
<h2 class="gb-headline gb-headline-996293b5 gb-headline-text">Step-by-Step Instructions</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>Apache JMeter has a detailed <a href="https://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.html" target="_blank" rel="noreferrer noopener">step-by-step instruction</a> for the distributed load testing. Even if you follow the instructions as-is, you might hit a few road blocks.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>This setup uses:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><strong>Windows 10 Pro </strong>(Controller) and <strong>Raspberry Pi OS</strong> (Worker Node)</li><li><strong>Java 15 </strong>in Windows 10 Pro and <strong>Java 11 </strong>in Raspberry Pi (this was intentional)</li><li><strong>JMeter 5.4 </strong>in both the OS</li></ul>
<!-- /wp:list -->

<!-- wp:image {"id":7964,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-29.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-29.png" alt="Distributed Load Testing in JMeter using Raspberry Pi 400" class="wp-image-7964"/></a><figcaption>Distributed Load Testing in JMeter using Raspberry Pi 400</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Here are the checkpoints before you start:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Controller and Worker Node must be in the same subnet</li><li>Firewall must be turned off and the required ports are opened</li><li>Java and JMeter version must be identical in all the nodes</li><li>Setup SSL for RMI</li></ul>
<!-- /wp:list -->

<!-- wp:generateblocks/headline {"uniqueId":"025aacec","element":"h3"} -->
<h3 class="gb-headline gb-headline-025aacec gb-headline-text">How to find whether the nodes are in same subnet?</h3>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>My controller node has 192.168.0.18 and my worker node has 192.168.0.37. To find the IP address and subnet mask in Windows, you can use <code>ipconfig</code> in command prompt and in Raspberry Pi, issue <code>ifconfig</code> in terminal.</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Go to https://www.meridianoutpost.com/resources/etools/network/two-ips-on-same-network.php </li><li>Enter both the IP addresses and subnet mask</li><li>Hit <strong>Calculate </strong></li></ul>
<!-- /wp:list -->

<!-- wp:image {"align":"center","id":7965,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-30.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-30.png" alt="Check for subnets" class="wp-image-7965"/></a><figcaption>Check for subnets</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:generateblocks/headline {"uniqueId":"c6d96e36"} -->
<h2 class="gb-headline gb-headline-c6d96e36 gb-headline-text">How it works?</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>Here is the pictorial representation of how it works. On a very high level, controller node instructs the worker node(s) by sending the JMeter file. Worker node will execute the threads as per the test plan design. Threads will hit the target application.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>From Controller Node, you can view the results, execution status, and errors/exceptions. You can start the test either using the CLI mode (recommended) or Non-CLI mode.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7966,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/Distributed-Load-Testing-JMeter-2.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/Distributed-Load-Testing-JMeter-2.png" alt="Distributed Load Testing Setup" class="wp-image-7966"/></a><figcaption>Distributed Load Testing Setup</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:generateblocks/headline {"uniqueId":"e19fe5f1"} -->
<h2 class="gb-headline gb-headline-e19fe5f1 gb-headline-text">Let's get started</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:generateblocks/headline {"uniqueId":"4aad026d","element":"h3"} -->
<h3 class="gb-headline gb-headline-4aad026d gb-headline-text">SSL Setup</h3>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>By default, JMeter uses SSL for RMI. If you want to disable SSL, you can do that in the properties file. Uncomment below property in <code>jmeter.properties</code> or configure it in <code>user.properties</code>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><em><code>server.rmi.ssl.disable=false</code></em></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>But we are going to setup SSL for RMI in this experiment. It is pretty straight forward.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Go to <code>bin</code> folder in Controller node, launch <code>create-rmi-keystore.bat</code> (Windows OS) or <code>create-rmi-keystore.sh</code> for macOS/Linux. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Keep answering the questions as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7968,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-32.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-32-1024x431.png" alt="Create RMI Key Store" class="wp-image-7968"/></a><figcaption>Create RMI Key Store</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>If you get the below error, it means already key is present.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7967,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-31.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-31-1024x51.png" alt="Create RMI Key Store" class="wp-image-7967"/></a><figcaption>Create RMI Key Store</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>After successful generation, you could see the <code>rmi_keystore.jks</code> file in <code>bin</code> folder. Copy this file into all the Worker nodes (bin folder).</p>
<!-- /wp:paragraph -->

<!-- wp:generateblocks/headline {"uniqueId":"c70f5e1d","element":"h4"} -->
<h4 class="gb-headline gb-headline-c70f5e1d gb-headline-text">Important Notes:</h4>
<!-- /wp:generateblocks/headline -->

<!-- wp:list -->
<ul><li>The default passphrase for the keystore is <code>changeit</code></li><li>Make sure you change the alias name in the properties file <code><em>server.rmi.ssl.keystore.alias=</em>qainsights</code></li><li>Below are the default RMI configuration properties</li></ul>
<!-- /wp:list -->

<!-- wp:code -->
<pre class="wp-block-code"><code># Configuration of Secure RMI connection
#
# Type of keystore : JKS
#server.rmi.ssl.keystore.type=JKS
#
# Keystore file that contains private key
#server.rmi.ssl.keystore.file=rmi_keystore.jks
#
# Password of keystore
#server.rmi.ssl.keystore.password=changeit
#
# Key alias
#server.rmi.ssl.keystore.alias=rmi
#
# Type of truststore : JKS
#server.rmi.ssl.truststore.type=JKS
#
# Keystore file that contains certificate
#server.rmi.ssl.truststore.file=rmi_keystore.jks
#
# Password of truststore
#server.rmi.ssl.truststore.password=changeit
#
# Set this if you don't want to use SSL for RMI
#server.rmi.ssl.disable=false</code></pre>
<!-- /wp:code -->

<!-- wp:generateblocks/headline {"uniqueId":"2dd5380b"} -->
<h2 class="gb-headline gb-headline-2dd5380b gb-headline-text">Controller Node JMeter Setup</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>Now it's time to configure the controller node JMeter setup.</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Open <code>jmeter.properties</code> in your favorite editor</li><li>Find <code>remote_hosts</code> property</li><li>Add all the worker nodes IP addresses separated by comma</li><li>E.g. <code>remote_hosts=192.168.0.37</code></li></ul>
<!-- /wp:list -->

<!-- wp:generateblocks/headline {"uniqueId":"4ae8297c"} -->
<h2 class="gb-headline gb-headline-4ae8297c gb-headline-text">Worker Node(s) JMeter Setup</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>You can either SSH or take remote your worker node(s). In this case, I am using VNC Viewer to take remote my Raspberry Pi 400.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Paste the <code>rmi_keystore.jks</code> file in the <code>bin</code> folder.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Launch <code>jmeter-server</code> using the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>./jmeter -s -Djava.rmi.server.hostname=192.168.0.37</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>or</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>./jmeter-server -Djava.rmi.server.hostname=192.168.0.37</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If everything goes well, you will get the below output. Now your remote machine is ready to take instructions.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>Created remote object: UnicastServerRef2 &#91;liveRef: &#91;endpoint:192.168.0.37:45971,SSLRMIServerSocketFactory(host=/192.168.0.37, keyStoreLocation=rmi_keystore.jks, type=JKS, trustStoreLocation=rmi_keystore.jks, type=JKS, alias=rmi),SSLRMIClientSocketFactory(keyStoreLocation=rmi_keystore.jks, type=JKS, trustStoreLocation=rmi_keystore.jks, type=JKS, alias=rmi),objID:&#91;-79d83a36:1768d7268d4:-7fff, 2199432749051545387]]]</code></pre>
<!-- /wp:code -->

<!-- wp:heading -->
<h2 id="h-back-to-controller-node">Back to Controller Node</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Let's go back to Controller Node (Windows 10 Pro). We have already configured the remote hosts. Go to <code>bin</code> folder in command prompt or terminal, launch the JMeter using the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>jmeter -Djava.rmi.server.hostname=192.168.0.18</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>This will launch JMeter in non-CLI mode.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Open the test plan which you are going to execute.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Do not hit <strong>Run</strong> button. Instead, go to <strong>Run &gt; Remote Start</strong> where you could see the remote hosts IP addresses. Select the IP address or you can have remote start all using <strong>Run &gt; Remote Start All.</strong></p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7971,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-35.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-35.png" alt="Remote Start" class="wp-image-7971"/></a><figcaption>Remote Start</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Below is the sample test plan.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7970,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-34.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-34.png" alt="" class="wp-image-7970"/></a></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>In the worker node, you will see the below output stating that testing has been started.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7972,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-36.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-36-1024x143.png" alt="Successful Remote Start" class="wp-image-7972"/></a><figcaption>Successful Remote Start</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>During the test execution, you can monitor the stats in <strong>Summary Report </strong>in JMeter (Controller Node).</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7973,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-37.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-37-1024x326.png" alt="Summary Report" class="wp-image-7973"/></a><figcaption>Summary Report</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>After the execution is completed, you can stop the JMeter instance in your worker node or keep it running.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Congratulations! You just completed the distributed load testing.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Tip</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To remote start the test in CLI mode, use the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>jmeter -Djava.rmi.server.hostname=controller_IP -n -t Distributed-Load-Test-Demo.jmx -l RemoteRun.jtl -R worker_IP1,worker_IP2..</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Important Points</strong></p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>If you get any error during this process, make sure you check the <code>jmeter.log</code> files in Controller and Worker nodes.</li><li>Always launch the Controller and Worker node using <code>Djava.rmi.server.hostname</code> argument or configure <code>RMI_HOST_DEF=-Djava.rmi.server.hostname=</code>xxx.xxx.xxx.xxx in the <code>jmeter-server</code> file </li><li>Make sure all the nodes are in same subnet</li><li>Firewall is disabled</li><li>JMeter strips the response data to optimize the network usage</li><li>RMI cannot communicate across subnets without a proxy</li><li>More about <a href="https://docs.oracle.com/javase/8/docs/technotes/guides/rmi/javarmiproperties.html" target="_blank" rel="noreferrer noopener">RMI Properties</a></li></ul>
<!-- /wp:list -->

<!-- wp:generateblocks/headline {"uniqueId":"848a9e5b"} -->
<h2 class="gb-headline gb-headline-848a9e5b gb-headline-text">Final Words</h2>
<!-- /wp:generateblocks/headline -->

<!-- wp:paragraph -->
<p>The initial effort to set up the distributed load testing might take time depends on the issues you face. The similar approach you can use it in AWS, GCP, Azure or other cloud providers.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If you want to save this overhead, you can also opt for ready-made JMeter-on-Cloud solutions. It all depends on how much you want to spend on the setup, maintenance, and spend time and effort on securing the nodes with the latest patches, versions etc.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>You can also leverage Docker, Kubernetes tech for the distribution or write a shell script to automate most of the moving parts.</p>
<!-- /wp:paragraph -->