Run JMeter Tests in k3s Civo Kube100
Tags: kubernetes, jmeter, testing, k3s
Published: 2020-08-10 19:53:34

<!-- wp:paragraph -->
<p>Hey perf guys, as you aware, in recent times I have been working on Kubernetes. Even I got my first certification in Kubernetes - CKAD. You can check it out <a href="https://qainsights.com/how-i-successfully-cleared-certified-kubernetes-application-developer-ckad-exam-in-5-weeks/" target="_blank" rel="noreferrer noopener">how I cracked CKAD exam in my last blog post</a>. Now I am test-driving <a href="https://found.ee/InlG" target="_blank" rel="noreferrer noopener">Civo's Kube100</a> platform. From the knowledge I acquired from the CKAD exam helped me to implement k3s to run JMeter tests in Civo's Jub100. This blog post will guide you how to run JMeter tests in k3s Civo Kube100 platform.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Before we dive into JMeter, let me give you an overview about Civo and Kube100.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>What is k3s?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>k3s is a lightweight Kubernetes. It is a certified Kubernetes distribution built for IoT &amp; Edge computing.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>About Civo</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Civo is a cloud hosting platform based out of UK. They primarily work on cloud and Kubernetes. Civo created world's first k3s-powered, managed Kubernetes service.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Using Civo, you can spin up Kubernetes cluster in couple minutes and make use of Civo's features from the dashboard. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Features such as node management, DDoS protection, custom firewalls, bandwidth pooling etc.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Right now, you can <a href="https://found.ee/InlG" target="_blank" rel="noreferrer noopener">test-drive Civo's Kubernetes service</a> using the beta program called <a href="https://found.ee/InlG" target="_blank" rel="noreferrer noopener">Kube100</a>. During the beta, you will get free credits of $70 per month.</p>
<!-- /wp:paragraph -->

<!-- wp:buttons -->
<div class="wp-block-buttons"><!-- wp:button {"gradient":"light-green-cyan-to-vivid-green-cyan"} -->
<div class="wp-block-button"><a class="wp-block-button__link has-background has-light-green-cyan-to-vivid-green-cyan-gradient-background" href="" target="_blank" rel="noreferrer noopener">Click here to join KUBE100 beta program</a></div>
<!-- /wp:button --></div>
<!-- /wp:buttons -->

<!-- wp:heading -->
<h2>Prerequisites</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Before we being seeing the implementation of JMeter in Kubernetes cluster, let us install the following prerequisites.</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><a href="https://found.ee/InlG" target="_blank" rel="noreferrer noopener">Civo Account</a> (free to join - if your application approved, you will get $70 per month in form of free credits)</li><li><a href="https://github.com/civo/cli" target="_blank" rel="noreferrer noopener">Civo CLI</a> (optional) </li><li><a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/" target="_blank" rel="noreferrer noopener">kubectl</a><ul><li><code>kubectl</code> allows you to run commands against Kubernetes clusters.</li></ul></li><li><a href="https://kubernetes.io/docs/tasks/tools/install-minikube/" target="_blank" rel="noreferrer noopener">Minikube</a> (optional)<ul><li>if you want to play with single-node Kubernetes cluster in your laptop or VM</li></ul></li><li>Favorite command line tool<ul><li>I prefer Windows Terminal</li></ul></li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Let us get started.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>Creating Kubernetes Cluster in Civo</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Login into your Civo account and click on <strong>Kubernetes</strong> and then click on <strong>Create cluster </strong>as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7483,"sizeSlug":"large"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-1024x470.png" alt="Run JMeter Tests in k3s Civo Kube100" class="wp-image-7483"/><figcaption>Run JMeter Tests in k3s Civo Kube100</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Enter the following as mentioned in the screenshot:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Cluster name</li><li>Node count</li><li>Node size as <strong>Extra Small</strong></li></ul>
<!-- /wp:list -->

<!-- wp:image {"align":"center","id":7485,"sizeSlug":"large"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-1-1024x437.png" alt="Cluster Config" class="wp-image-7485"/><figcaption>Cluster Config</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>To store the JMeter results in the persistent volume, we will be using Rancher's Longhorn. It is available in the Civo's marketplace. Just scroll down a bit, click on <strong>Storage</strong> and then select <strong>Longhorn</strong>. Click on <strong>Create </strong>button to create the cluster. </p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7486,"sizeSlug":"large"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-2-1024x411.png" alt="" class="wp-image-7486"/></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Creating the cluster will take couple minutes or less. After successful creation, you need to download the Kubeconfig file as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7488,"sizeSlug":"large"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-3.png" alt="Kubeconfig " class="wp-image-7488"/><figcaption>Kubeconfig</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>Store the Kubeconfig file in a secured location. It contains cluster admin credentials.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>Accessing nodes using kubectl</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Launch your favorite command line tool. I am using Windows Terminal. First step is to configure the Kubeconfig file, so that you can access the nodes.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Use the below command in Windows to configure the Kubeconfig.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>setx KUBECONFIG 'C:\Tools\civo-0.6.19-windows-amd64\civo-jmeter-cluster-kubeconfig'</code></p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7489,"sizeSlug":"large"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-4-1024x81.png" alt="Kubeconfig Set" class="wp-image-7489"/><figcaption>Kubeconfig Set</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>If you are in Linux, you can use the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>export KUBECONFIG="path_to_config_file"</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Let us validate whether we can access the Kubernetes clusters or not by issuing the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl get nodes</code></p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7493,"sizeSlug":"large"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-5.png" alt="Get Nodes" class="wp-image-7493"/><figcaption>Get Nodes</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:heading -->
<h2>Deploy Pods</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Before we start creating pods for JMeter, we need to store the results of our tests to the persistent volume. Let us leverage <a href="https://rancher.com/products/longhorn/" target="_blank" rel="noreferrer noopener">Rancher's Longhorn </a>which is available in the Civo marketplace.</p>
<!-- /wp:paragraph -->

<!-- wp:buttons -->
<div class="wp-block-buttons"><!-- wp:button {"gradient":"vivid-cyan-blue-to-vivid-purple"} -->
<div class="wp-block-button"><a class="wp-block-button__link has-background has-vivid-cyan-blue-to-vivid-purple-gradient-background" href="https://found.ee/InlG" target="_blank" rel="noreferrer noopener">Join Kube100 and receive $70 monthly free credits</a></div>
<!-- /wp:button --></div>
<!-- /wp:buttons -->

<!-- wp:heading {"level":3} -->
<h3>Persistent Volume</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>First step is to create a persistent volume using the below yml. The name of the persistent volume is <code>jmeter-pv</code>. Copy the below yml config and save it to your local and apply the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl create -f .\pv.yml</code></p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>apiVersion: v1
kind: PersistentVolume
metadata:
  name: jmeter-pv
  namespace: jmeter
  labels:
    name: jmeter-data
    type: longhorn
spec:
  capacity:
    storage: 1G
  volumeMode: Filesystem
  storageClassName: longhorn
  accessModes:
    - ReadWriteOnce
  csi:
    driver: io.rancher.longhorn
    fsType: ext4
    volumeAttributes:
      numberOfReplicates: '2'
      staleReplicaTimeout: '20'
    volumeHandle: jmeter-data</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>To validate the persistent volume, issue the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl get pv</code></p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7495,"sizeSlug":"large"} -->
<figure class="wp-block-image size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-6.png" alt="" class="wp-image-7495"/></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":3} -->
<h3>Persistent Volume Claim</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Next step is to claim the created volume to use it in the JMeter pod. To claim the persistent volume, copy the below yml and save it to your local and issue the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl create -f .\pvc.yml</code></p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jmeter-pv-claim
  labels:
    type: longhorn
    app: jmeter
spec:
  storageClassName: longhorn
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>To validate the persistent volume claim, issue the below command and make sure the status is bound.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl get pvc</code></p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7496,"sizeSlug":"large"} -->
<figure class="wp-block-image size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-7-1024x54.png" alt="Get PVC" class="wp-image-7496"/><figcaption>Get PVC</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":3} -->
<h3>JMeter Pod</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>To run our JMeter tests we are going to make use of the following image <code>justb4/jmeter</code>. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Copy the below yml and save it to your local and issue the below command to run the JMeter tests.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl create -f .\jmeter-pod.yml</code></p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: jmeter
  name: jmeter
spec:
  containers:
  - image: justb4/jmeter
    name: jmeter
    command: &#91;"sh","-c","jmeter -n -t /opt/apache-jmeter-5.1.1/bin/examples/CSVSample.jmx -l /opt/results/mylog.log"]
    volumeMounts:
      - name:  jmeter-pv-claim
        mountPath:  "/opt/results"
  volumes:
  - name: jmeter-pv-claim
    persistentVolumeClaim:
      claimName: jmeter-pv-claim
  restartPolicy: Never
status: {}</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Above pod configuration executes JMeter test plan which is located in the <code>examples</code> folder and then stores the results to the persistent volume in the path <code>/opt/results</code>.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To validate the pod is running successfully or not, issue the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl get po</code></p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center","id":7499,"sizeSlug":"large"} -->
<div class="wp-block-image"><figure class="aligncenter size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-8.png" alt="Get Pods" class="wp-image-7499"/><figcaption>Get Pods</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>As you see in the above screenshot, <code>jmeter</code> pod ran successfully and completed the tasks.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>After the completion, you can view the logs of the pod by issuing the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl logs jmeter</code></p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7500,"sizeSlug":"large"} -->
<figure class="wp-block-image size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-9-1024x227.png" alt="Get Logs" class="wp-image-7500"/><figcaption>Get Logs</figcaption></figure>
<!-- /wp:image -->

<!-- wp:heading {"level":3} -->
<h3>Download the JMeter raw results</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>To view or download the JMeter results which are stored in <code>/opt/results</code> folder in the persistent storage, we are going to make use of busybox pod.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Copy the below yaml and save it to local and then issue the below command to create the pod.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl create -f .\busybox.yml</code></p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>apiVersion: v1
kind: Pod
metadata:
  labels:
    run: bb
  name: bb
spec:
  containers:
  - image: busybox
    name: bb
    command: &#91;"sh","-c","sleep 3600"]
    volumeMounts:
      - name:  jmeter-pv-claim
        mountPath:  "/opt/results"
  volumes:
  - name: jmeter-pv-claim
    persistentVolumeClaim:
      claimName: jmeter-pv-claim         
  restartPolicy: Never</code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>To validate the pod, issue the below command. It should be <code>Running</code> status.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7501,"sizeSlug":"large"} -->
<figure class="wp-block-image size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-10.png" alt="Get Pods" class="wp-image-7501"/><figcaption>Get Pods</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>To validate whether the results are stored in the persistent volume we need to log in into the <code>bb</code> pod.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To login into the busybox pod, issue the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl exec bb -it -- sh</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>This will enable you to peek into the pod to run commands. Type <code>cat /opt/results/mylog.log</code> to view the raw results as shown below.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7502,"sizeSlug":"large"} -->
<figure class="wp-block-image size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-11-1024x208.png" alt="Exec Pod" class="wp-image-7502"/><figcaption>Exec Pod</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>To come out of the <code>bb</code> pod, type <code>exit</code> command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To copy the contents, issue the below command.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>kubectl cp bb:/opt/results/mylog.log jmeter-results</code></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>This command will copy the log file and create a new folder <code>jmeter-results</code> in your local and paste the log file into it.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>To view the results in JMeter, launch JMeter and add the desired listener and browse the results (make sure you change the extension to .jtl)</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":7509,"sizeSlug":"large"} -->
<figure class="wp-block-image size-large"><img src="https://qainsights.com/wp-content/uploads/2020/08/image-12-1024x273.png" alt="JMeter Results" class="wp-image-7509"/><figcaption>JMeter Results</figcaption></figure>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>? Congratulations! You successfully completed running JMeter test plan in Kubernetes in Civo ?</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>What we have done is just the tip of the iceberg. You can run multiple pods or deployments to inject more load. Or you could integrate Grafana to view the stats instantly. </p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Points to note:</h3>
<!-- /wp:heading -->

<!-- wp:list -->
<ul><li>If you use high power nodes, you can inject more threads by following JMeter best practices</li><li>It is possible to leverage distributed load testing, I will work on that eventually</li><li>By integrating with InfluxDB/Grafana/Prometheus, you will get the luxury of storing results for analysis</li><li>Affordable pricing and ease of maintenance</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>All the yaml files are available in my <a href="https://github.com/QAInsights/civo-jmeter-k3s" target="_blank" rel="noreferrer noopener">GitHub repo</a>. Please star and fork it.</p>
<!-- /wp:paragraph -->

<!-- wp:buttons -->
<div class="wp-block-buttons"><!-- wp:button {"gradient":"blush-bordeaux"} -->
<div class="wp-block-button"><a class="wp-block-button__link has-background has-blush-bordeaux-gradient-background" href="https://github.com/QAInsights/civo-jmeter-k3s" target="_blank" rel="noreferrer noopener">Civo JMeter GitHub repo</a></div>
<!-- /wp:button --></div>
<!-- /wp:buttons -->

<!-- wp:paragraph -->
<p>Please let me know if you hit any road block, happy to help you out.</p>
<!-- /wp:paragraph -->