Right Tail Outlier Detector in JMeter
Tags: testing, tutorial, webperf, performance
Published: 2021-02-05 14:40:27

<!-- wp:paragraph -->
<p>In this blog post we are going to see about Right Tail Outlier Detector in JMeter. Analyzing results is one of the critical phase in any performance testing project. Most of the results you encounter might have outliers. In commercial tools such as NeoLoad has a built-in feature to remove the noise. In <a href="https://qainsights.com" target="_blank" rel="noreferrer noopener">JMeter</a>, you can isolate the outliers using the Right Tail Outlier Detector plugin.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-installing-right-tail-outlier-detector-in-jmeter">Installing Right Tail Outlier Detector in JMeter</h2>
<!-- /wp:heading -->

<!-- wp:heading {"level":3} -->
<h3 id="h-prerequisites">Prerequisites</h3>
<!-- /wp:heading -->

<!-- wp:list -->
<ul><li>Stable version of JMeter </li><li>Plugins Manager</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>In this demonstration, I am using JMeter 5.4.1 and Plugins Manager 1.6. Download the Plugins Manager jar from <a href="https://jmeter-plugins.org/install/Install/" target="_blank" rel="noreferrer noopener">here</a> and place it into <code>JMETER_HOME\lib\ext</code> folder.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>(Re)Start JMeter to install the Right Tail Outlier Detector plugin.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Launch <strong>Plugins Manager</strong> by clicking on the icon at the top right.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Search for "<strong>Right Tail</strong>" under the "<strong>Available Plugins</strong>" as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8210,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/02/image.png"><img src="https://qainsights.com/wp-content/uploads/2021/02/image.png" alt="Right Tail Outlier Detector in JMeter" class="wp-image-8210"/></a><figcaption>Right Tail Outlier Detector in JMeter</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Hit Apply Changes and Restart JMeter. This will download the relevant jars and place it in the appropriate locations and restart JMeter.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-usage">Usage</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Launch JMeter and add the <strong>Right Tail Outlier Detection</strong> element which is located under<strong> Non-Test Elements</strong> as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8211,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/02/image-1.png"><img src="https://qainsights.com/wp-content/uploads/2021/02/image-1.png" alt="Add Right Tail Outlier Detection" class="wp-image-8211"/></a><figcaption>Add Right Tail Outlier Detection</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>You can see the element configuration as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8212,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/02/image-2.png"><img src="https://qainsights.com/wp-content/uploads/2021/02/image-2-1024x694.png" alt="Right Tail Outlier Detection Configuration" class="wp-image-8212"/></a><figcaption>Right Tail Outlier Detection Configuration</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:heading -->
<h2 id="h-what-is-an-outlier">What is an Outlier?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>From performance test result perspective, an Outlier is a data point which is significantly different from the other data points. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>E.g. following are the response time of a page load during the test run. The data point at 17:19:00 has 1.061 seconds which is different from the other data points. Here the outlier is 1.061 seconds.</p>
<!-- /wp:paragraph -->

<!-- wp:table {"backgroundColor":"subtle-pale-blue","className":"is-style-stripes"} -->
<figure class="wp-block-table is-style-stripes"><table class="has-subtle-pale-blue-background-color has-background"><tbody><tr><td class="has-text-align-center" data-align="center"><strong>Time Stamp</strong></td><td class="has-text-align-center" data-align="center"><strong>Response Time (s)</strong></td></tr><tr><td class="has-text-align-center" data-align="center">17:15:00</td><td class="has-text-align-center" data-align="center">0.05</td></tr><tr><td class="has-text-align-center" data-align="center">17:16:00</td><td class="has-text-align-center" data-align="center">0.07</td></tr><tr><td class="has-text-align-center" data-align="center">17:17:00</td><td class="has-text-align-center" data-align="center">0.06</td></tr><tr><td class="has-text-align-center" data-align="center">17:18:00</td><td class="has-text-align-center" data-align="center">0.045</td></tr><tr><td class="has-text-align-center" data-align="center">17:19:00</td><td class="has-text-align-center" data-align="center">1.061</td></tr><tr><td class="has-text-align-center" data-align="center">17:20:00</td><td class="has-text-align-center" data-align="center">0.046</td></tr></tbody></table><figcaption>Sample Data with an outlier</figcaption></figure>
<!-- /wp:table -->

<!-- wp:heading -->
<h2 id="h-about-right-tail-outlier-detector-plugin">About Right Tail Outlier Detector Plugin</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>This plugin uses Tukey's technique to find the upper bounds using two options: 1.5 and 3 Tukey's constant <strong>k</strong>. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>k 1.5 will identify all the outliers and k 3 will identify only the extreme values.</p>
<!-- /wp:paragraph -->

<!-- wp:buttons {"align":"center"} -->
<div class="wp-block-buttons aligncenter"><!-- wp:button {"textColor":"white","gradient":"vivid-cyan-blue-to-vivid-purple"} -->
<div class="wp-block-button"><a class="wp-block-button__link has-white-color has-vivid-cyan-blue-to-vivid-purple-gradient-background has-text-color has-background" href="https://github.com/rbourga/jmeter-plugins-2" target="_blank" rel="noreferrer noopener">Right Tail Outlier Detector Plugin Source Code</a></div>
<!-- /wp:button --></div>
<!-- /wp:buttons -->

<!-- wp:paragraph -->
<p>To detect the outliers, first you need the performance test results either in jtl, csv or tsv format.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>I have the sample test result which I ran against my local app. First let us detect all the outliers using k 1.5.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Select 1.5 (detect all outliers) and browse the results file as shown below and then click on <strong>Detect</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8214,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/02/image-4.png"><img src="https://qainsights.com/wp-content/uploads/2021/02/image-4-1024x545.png" alt="Outliers Detection" class="wp-image-8214"/></a><figcaption>Outliers Detection</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>For my test result, I have got 5 Outliers and the Outliers and Trimmed files have been generated in the <strong>bin </strong>folder.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8215,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/02/image-5.png"><img src="https://qainsights.com/wp-content/uploads/2021/02/image-5.png" alt="Outliers and Trimmed Files" class="wp-image-8215"/></a><figcaption>Outliers and Trimmed Files</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>You could discard the Outliers file and consider only the Trimmed file for further analysis. In case if you want to analyze the root cause of potential performance problems, you can consider Outliers files.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>You can also see the Upper fence and the Trimmer count in the table view. To save the table data, click on <strong>Save Table Data.</strong></p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":8216,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2021/02/image-6.png"><img src="https://qainsights.com/wp-content/uploads/2021/02/image-6-1024x251.png" alt="k 1.5 Detection" class="wp-image-8216"/></a><figcaption>k 1.5 Detection</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>For k 3, there are no Outliers detected for my test results, as it will detect only the extrement values.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":8218,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2021/02/image-8.png"><img src="https://qainsights.com/wp-content/uploads/2021/02/image-8-1024x380.png" alt="k 3 detection" class="wp-image-8218"/></a><figcaption>k 3 detection</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:heading -->
<h2 id="h-command-line-usage">Command Line Usage</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>You can leverage command line to identify the outliers on the fly. Issue the below command to detect the outliers for k 1.5.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>java -jar lib\cmdrunner-2.2.jar --tool RightTailOutlierDetector --input-file bin\Run2.jtl --tukey-k 1.5</code></pre>
<!-- /wp:code -->

<!-- wp:image {"id":8219,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2021/02/image-9.png"><img src="https://qainsights.com/wp-content/uploads/2021/02/image-9-1024x189.png" alt="Command Line Usage" class="wp-image-8219"/></a><figcaption>Command Line Usage</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading -->
<h2 id="h-conclusion">Conclusion</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Detecting outliers helps you to analyze the performance test results. Outlier detection should be your first step in your checklist. This plugin saves time and helps you to detect in few clicks. Do you use any other method to detect the outliers? Please let me know in the comments.</p>
<!-- /wp:paragraph -->