JMeter Integration with InfluxDB and Grafana using Docker
Tags: docker, performance, testing, tutorial
Published: 2020-12-28 16:44:15

<!-- wp:paragraph -->
<p>As a part of my <a href="https://www.youtube.com/playlist?list=PLJ9A48W0kpRIjLkZ32Do9yDZXnnm7_uj_" target="_blank" rel="noreferrer noopener">Learn JMeter Series in YouTube</a>, I was working on JMeter Integration with InfluxDB and Grafana. During the study phase, I have learned few more insights about the integration. This blog post focuses on JMeter Integration with InfluxDB and Grafana using Docker. You can check my <a href="https://qainsights.com/jmeter-prometheus-and-grafana-integration/" target="_blank" rel="noreferrer noopener">Prometheus Integration with JMeter in here</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Before we get started, let us understand what each component is and its purpose. I have also created a simple diagram to understand the integration.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-what-is-influxdb"><strong>What is InfluxDB?</strong></h2>
<!-- /wp:heading -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>InfluxDB is an open-source time series database developed by InfluxData. It is written in Go and optimized for fast, high-availability storage and retrieval of time series data in fields such as operations monitoring, application metrics, Internet of Things sensor data, and real-time analytics.</p><cite>from Wikipedia</cite></blockquote>
<!-- /wp:quote -->

<!-- wp:heading -->
<h2 id="h-what-is-grafana">What is Grafana?</h2>
<!-- /wp:heading -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>Grafana is a multi-platform open source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources, Grafana Enterprise version with additional capabilities is also available. It is expandable through a plug-in system.</p><cite>from Wikipedia</cite></blockquote>
<!-- /wp:quote -->

<!-- wp:heading -->
<h2 id="h-what-is-docker">What is Docker?</h2>
<!-- /wp:heading -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.</p><cite>from Wikipedia</cite></blockquote>
<!-- /wp:quote -->

<!-- wp:image {"id":7978,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/JMeter-Integration-with-InfluxDB-and-Grafana-using-Docker.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/JMeter-Integration-with-InfluxDB-and-Grafana-using-Docker-1024x512.png" alt="JMeter Integration with InfluxDB and Grafana using Docker" class="wp-image-7978"/></a><figcaption>JMeter Integration with InfluxDB and Grafana using Docker</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading -->
<h2 id="h-prerequisites">Prerequisites</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Following are the prerequisites required for the integration:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>Docker</li><li>JMeter 5.4</li><li>Sample JMeter Test Plan</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>This integration has been tested in Windows 10 Pro. But it should work in other OS using the similar commands.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-let-s-get-started">Let's get started</h2>
<!-- /wp:heading -->

<!-- wp:heading {"level":3} -->
<h3 id="h-influx-db-up-and-running-in-docker">Influx DB up and running in Docker</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>First let's start with InfluxDB setup in Docker. This integration uses the InfluxDB v1.8.3.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Download the InfluxDB 1.8.3 <a href="https://portal.influxdata.com/downloads/">in here</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Extract the contents. We are just going to use the <code>influxdb.conf</code> file in our Docker container.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Open the terminal and run the below command to spin the InfluxDB container.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>docker run -p 8000:8086 -d -v C:\Tools\influxdb-1.8.3_windows_amd64\influxdb-1.8.3-1:/var/lib/influxdb influxdb:1.8 -config /var/lib/influxdb/influxdb.conf</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Then enter <code>docker ps</code> to see the running containers list. You will get the below output.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7979,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-38.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-38-1024x43.png" alt="" class="wp-image-7979"/></a></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Launch http://localhost:8000 in your favor browser, you will get </p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">404 page not found</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>That is fine. We are going to create a database by executing commands into the docker container.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Enter the below command to execute <code>sh</code> into the running container.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>docker exec -it &lt;container-id&gt; sh</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>After logging into the InfluxDB container, enter <code>influx</code> command which will logins into InfluxDB Shell as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7980,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-39.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-39.png" alt="" class="wp-image-7980"/></a></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Enter the below command to create a database called <code>jmeter</code>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>create database jmeter</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Then, enter <code>show databases</code> command which will list out all the available databases.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Let us create an admin user for InfluxDB by entering the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>CREATE USER admin WITH PASSWORD 'admin123' WITH ALL PRIVILEGES</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Type <code>exit</code> twice to come out of the container.</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3 id="h-grafana-up-and-running">Grafana up and running</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Creating Grafana container is straightforward. Enter the below command and hit enter.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>docker run -d --name=grafana -p 4000:3000 grafana/grafana</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Launch http://localhost:4000 in your favorite browser. The default credential is <code>admin</code> and <code>admin</code>. Upon first time login, it will prompt you to change the password.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7981,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-40.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-40-1024x564.png" alt="Grafana Login" class="wp-image-7981"/></a><figcaption>Grafana Login</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Hover over <strong>Settings</strong> icon, click on <strong>Data Sources</strong> to add our InfluxDB container.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7982,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-41.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-41.png" alt="" class="wp-image-7982"/></a></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>In <strong>Configuration</strong>, click on <strong>Add data source </strong>button and then select <strong>InfluxDB.</strong></p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7983,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-42.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-42.png" alt="" class="wp-image-7983"/></a></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Fill the detailed as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7984,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-43.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-43.png" alt="" class="wp-image-7984"/></a></figure></div>
<!-- /wp:image -->

<!-- wp:list {"ordered":true} -->
<ol><li>Name of the connection</li><li>Query Language as <strong>InfluxQL</strong></li><li>InfluxDB connection with the port and <strong>Access </strong>as <strong>Browser.</strong></li><li>Toggle off Auth options</li><li>Enter database name, username, password and HTTP method as jmeter, admin, admin123, and GET respectively.</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Click on <strong>Save</strong> <strong>&amp; Test </strong>button. If everything goes well, you will success message <code>Data source is working.</code></p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7985,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-44.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-44.png" alt="" class="wp-image-7985"/></a></figure></div>
<!-- /wp:image -->

<!-- wp:heading -->
<h2 id="h-jmeter-test-plan-set-up">JMeter Test Plan Set up</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Launch JMeter and open the test plan which you will be executing. Add a <strong>Backend Listener </strong>by right clicking on <strong>Thread Group &gt; Add &gt; Listener &gt; Backend Listener.</strong></p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7987,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-45.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-45.png" alt="" class="wp-image-7987"/></a></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Configure the listener as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7988,"sizeSlug":"large","linkDestination":"media"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-46.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-46-1024x382.png" alt="" class="wp-image-7988"/></a></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Make sure you selected the implementation as <code>InfluxdbBackendListenerClient</code> and <code>influxdbMetricsSender</code> as <code>org.apache.jmeter.visualizers.backend.influxdb.HttpMetricsSender</code>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Save your test plan and start the test in CLI mode.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Navigate to Grafana &gt; Explore, select the database and configure the query as shown below. You will get the performance stats realtime while the test is running.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7989,"sizeSlug":"large","linkDestination":"media"} -->
<figure class="wp-block-image size-large"><a href="https://qainsights.com/wp-content/uploads/2020/12/image-47.png"><img src="https://qainsights.com/wp-content/uploads/2020/12/image-47-1024x348.png" alt="" class="wp-image-7989"/></a></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Here is the query to retrieve 95 percentile response time in Grafana.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>SELECT mean("pct95.0") FROM "jmeter" WHERE ("transaction" = 'all') AND $timeFilter GROUP BY time($__interval) fill(null)</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Now the integration is successful, you can start creating dashboards based on your needs.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>Important Points:</strong></p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>The default port of InfluxDB is 8086. As we are running it as docker container, I am exposing it at 8000.</li><li>The containers of Grafana and InfluxDB are ephemeral. If you kill the containers, you will lose everything. You can persist the data by mounting the volume.</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>You can stop the container using <code>docker container stop &lt;container_id&gt;</code>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>You can remove the container using <code>docker container rm &lt;container_id&gt;</code>.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2 id="h-final-words">Final Words</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>The integration of InfluxDB and Grafana is pretty simple and straightforward. Running them in Docker containers is even easier. This integration helps us in monitoring the performance data in real time and also in taking the necessary steps while test is running.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>You can set up the alerts, create dashboards, share dashboards, create powerful query and more in Grafana. </p>
<!-- /wp:paragraph -->