Correlation - The Hard Way in JMeter
Tags: tutorial, java, testing, performance
Published: 2022-08-29 14:47:55

<!-- wp:paragraph -->
<p>I have already posted many video tutorials about correlation in JMeter, LoadRunner, and other tools on my <a href="https://youtube.com/qainsights" target="_blank" rel="noreferrer noopener">YouTube channel</a>. But all the methods which I explained are abstracted and traditional. In this blog post, let us see how it can be done in a hard way. </p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>What is Correlation?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>There is no official term as <code>correlation</code> in JMeter ecosystem. It is a generic term used by performance engineers/testers. Also, entitling this blog post with a <code>correlation</code> word helps in SEO :) </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Correlation is a process of extracting a string from the response body, response header or basically anything from the response. After extracting the response, it can be stored in a variable for subsequent use. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Typical example would be a login session. Session IDs are unique and gibberish. Hard-coding it in the script is not an effective way of handling it. Because it may expire based on the web server-side properties.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>Correlation - Hard way in JMeter</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Let us start with extracting a <code>title</code> tag from the response using Groovy scripting by adding a <a href="https://www.youtube.com/playlist?list=PLJ9A48W0kpRIjLkZ32Do9yDZXnnm7_uj_" target="_blank" rel="noreferrer noopener">JSR223 Post processor</a> in JMeter. Below is the test plan tree.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":9703,"sizeSlug":"full","linkDestination":"media"} -->
<figure class="wp-block-image aligncenter size-full"><a href="https://qainsights.com/wp-content/uploads/2022/08/image-7.png"><img src="https://qainsights.com/wp-content/uploads/2022/08/image-7.png" alt="Correlation - The Hard Way in JMeter" class="wp-image-9703"/></a><figcaption>Correlation - The Hard Way in JMeter</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Here is the Groovy script.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>response = prev.getResponseDataAsString() //Extract the previous response

def extractTitle = /&lt;title&gt;(.+?)&lt;\/title&gt;/
def matcher = response =~ extractTitle

if (matcher.size() &gt;=1) {
    println matcher.findAll()&#91;0]&#91;1]
    vars.put("extractTitle",matcher.findAll()&#91;0]&#91;1])
}</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Here is the URL https://jpetstore-qainsights.cloud.okteto.net/jpetstore/actions/Catalog.action</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The first step is to read the HTTP response as a string using <code>prev.getResponseDataAsString()</code> </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>prev</code> is an API call which extracts the <a href="https://jmeter.apache.org/api/org/apache/jmeter/samplers/SampleResult.html" target="_blank" rel="noreferrer noopener"><strong>prev</strong>ious SampleResult</a>. Using the method <code>getResponseDataAsString()</code> we can extract the whole response as a string and store it in a variable.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The next two lines define our regular expression pattern and the matching conditions. <a href="https://groovy-lang.org/syntax.html#_slashy_string" target="_blank" rel="noreferrer noopener">Groovy</a> comes with powerful regular expression pattern matching.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>def extractTitle = /&lt;title&gt;(.+?)&lt;\/title&gt;/
def matcher = response =~ extractTitle</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>The next block checks for any matches of &gt;=1, then it will print the extracted string from the array list. Then, it will store the value to the variable <code>extractTitle</code> using the <code>vars.put</code> method.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>if (matcher.size() &gt;=1) {
    println matcher.findAll()&#91;0]&#91;1]
    vars.put("extractTitle",matcher.findAll()&#91;0]&#91;1])
}</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Here is the output:</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":9704,"sizeSlug":"full","linkDestination":"media"} -->
<figure class="wp-block-image aligncenter size-full"><a href="https://qainsights.com/wp-content/uploads/2022/08/image-8.png"><img src="https://qainsights.com/wp-content/uploads/2022/08/image-8.png" alt="JMeter Output" class="wp-image-9704"/></a><figcaption>JMeter Output</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>The above method is not effective for a couple of reasons. One, the array index to capture the desired string might be cumbersome for the complex response. Second, typically the pattern we use here is apt for the text response, not for the HTML response. For the complex HTML response, using the regular expression might not yield better performance. </p>
<!-- /wp:paragraph -->

<!-- wp:generateblocks/button-container {"uniqueId":"52fb0772","alignment":"center","isDynamic":true,"blockVersion":2} -->
<!-- wp:generateblocks/button {"uniqueId":"441b2b5e","hasUrl":true,"target":true,"hasIcon":true,"backgroundColor":"var(\u002d\u002dglobal-color-8)","backgroundColorHover":"var(\u002d\u002dbase-3)","textColor":"var(\u002d\u002dcontrast)","textColorHover":"#ffffff","paddingTop":"15","paddingRight":"20","paddingBottom":"15","paddingLeft":"20","gradientDirection":90,"gradientColorOne":"var(\u002d\u002dglobal-color-8)","gradientColorTwo":"var(\u002d\u002dcontrast-3)","blockVersion":2} -->
<a class="gb-button gb-button-441b2b5e" href="https://github.com/QAInsights/Learn-JMeter-Series/tree/master/Correlation" target="_blank" rel="noopener noreferrer"><span class="gb-icon"><svg aria-hidden="true" role="img" height="1em" width="1em" viewBox="0 0 256 512" xmlns="http://www.w3.org/2000/svg"><path fill="currentColor" d="M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z"></path></svg></span><span class="gb-button-text">GitHub Repo</span></a>
<!-- /wp:generateblocks/button -->
<!-- /wp:generateblocks/button-container -->

<!-- wp:heading -->
<h2>Using JSoup</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>To handle the HTML response effectively, it is better to use HTML parsers such as JSoup. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><em>JSoup is a Java library for working with real-world HTML. It provides a very convenient API for fetching URLs and extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors.</em></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Let us use <code><a href="https://qainsights.com/upload-files-to-aws-s3-in-jmeter-using-groovy/" target="_blank" rel="noreferrer noopener">Grab</a></code>, so that JMeter will download the dependencies on its own, else you need to download the JSoup jar and keep it in the <code>lib</code> or <code>ext</code> folder. </p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>import org.jsoup.Jsoup
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element
import org.jsoup.select.Elements

@Grab(group='org.jsoup', module='jsoup', version='1.15.2')

response = prev.getResponseDataAsString() // Extract response

Document doc = Jsoup.parse(response)
println doc.title()</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p><code>doc</code> object will parse the response and print the title to the command prompt in JMeter.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To print all the links and its text, the below code snippet will be useful.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>
import org.jsoup.Jsoup
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element
import org.jsoup.select.Elements

@Grab(group='org.jsoup', module='jsoup', version='1.15.2')

response = prev.getResponseDataAsString() // Extract response

Document doc = Jsoup.parse(response)
println doc.title()

// To print all the links and its text

Elements links = doc.body().getElementsByTag("a");
for (Element link : links) {
    String linkHref = link.attr("href");
    String linkText = link.text();
    println linkHref + linkText
}</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>To print all the list box elements and random list box values for the url (http://computer-database.gatling.io/computers/new), use the below code snippet.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>import org.jsoup.Jsoup
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element
import org.jsoup.select.Elements

@Grab(group='org.jsoup', module='jsoup', version='1.15.2')

response = prev.getResponseDataAsString() // Extract response
companyList = &#91;]
Random random = new Random()

Document doc = Jsoup.parse(response)

// To print all the list box elements

Elements lists = doc.body().select("select option")
for (Element list : lists) {
    println "Company is " + list.text()
    companyList.add(list.text())
}

// To print random list box element

println("The total companies are " + companyList.size())
println(companyList&#91;random.nextInt(companyList.size())])</code></pre>
<!-- /wp:code -->

<!-- wp:heading -->
<h2>Final Words</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>As you learned, by leveraging the <code>prev</code> API we can extract the response and then parse it using JSoup library or by writing desired regular expressions in a hard way without using the built-in elements such as Regular Expression Extractor or JSON Extractor and more. This approach might not save time, but it is worth learning this approach which comes handy in situations like interviews. </p>
<!-- /wp:paragraph -->