Scale out your Gatling tests with Jenkins
Published: 2020-06-29 20:45:12

## Table Of Contents
  * [Quickstart](#quickstart)
  * [Requirements](#requirements)
  * [Scaling out our testing] (#scaling-out)
  * [SBT Setup] (#sbt-setup)
  * [Gatling Simulation Setup] (#gatling-sim-setup)
    * [Gatling Runner class] (#gatling-runner)
    * [Load Simulation class] (#load-simulation)
  * [Jenkins Pipeline Setup] (#jenkins-pipeline)
    * [Jenkinsfile] (#jenkinsfile)
    * [Jenkins Job Setup] (#jenkins-job)


[Gatling](http://gatling.io) is a load testing tool that allows developers to programmatically define flexible and powerful testing scenarios, via its' inbuilt DSL. Through the use of Akka Actors, it is able to push large amounts of load from even a single host.

Having said that, Gatling is capable of overwhelming the resources on the single host it is run from, given enough incentive i.e., very large amounts of connections, complex scenarios, large data feeders, etc. Once you've reached that point one of your options is to scale out your Gatling instances, thereby splitting the load between them. 

Although this can be done manually, running the tests and collecting and collating the data can be tedious and error-prone. There is a basic [scaling out] (https://gatling.io/docs/current/cookbook/scaling_out) procedure defined on the Gatling website but we can do better.
We will look at automating this procedure via _Jenkins Pipelines_ to a high degree, allowing you to spend more time testing and less time performing boilerplate manoeuvers to get to your test results.

### Quickstart <a name="quickstart"></a>
* Fork http://github.com/imavroukakis/gatling-scale-out/
* Edit `Jenkinsfile` to match your configuration and requirements
* Create a new Jenkins _MultiBranch Pipeline_ on the forked repo
* **Approve** `staticMethod java.lang.Math round double` in Jenkins
* Load test!

## Requirements <a name="requirements"></a>
* Jenkins v2.219 or higher on Linux, Pipeline capable and with the following plugins
  * SBT plugin
  * Github Pipeline plugin
  * AdoptOpenJDK plugin (or other JDK provider)
  * Build Name and Description Setter
  * Workspace Cleanup Plugin
* Linux Jenkins Workers
* A smattering of Scala
* Some familiarity with Gatling


## Scaling out our testing <a name="scaling-out"></a>

The approach we follow is this:

* Create a packaged version of our load tests via the SBT Pack plugin 
* Run our packaged app in a Jenkins Pipeline, supplying the required command-line parameters for users-per-second and test duration
* Parallel out tests to Jenkins nodes (if the users-per-second rate is higher than some limit)
* Collect, process and archive results from the Jenkins nodes

Let's start by taking a look at the code. We begin with our SBT build script.

### SBT setup <a name="sbt-setup"></a>

We are using the SBT Pack plugin, to create our packaged application. 
Create a file called `plugins.sbt` inside the `project` directory and add the following line:
```
addSbtPlugin("org.xerial.sbt" % "sbt-pack" % "0.12")
```
This will pull down the Pack plugin and make its' tasks available to SBT.

Our SBT build file is as follows
{% gist https://gist.github.com/imavroukakis/99289df78baa7f27e93af8d5e5fca0d4 file=build.sbt %}

At line 22, are the JVM options the application will run with, taken from the defaults recommended by Gatling. 
```
packJvmOpts := Map("load-test" -> 
Seq("-Xms2G -Xmx2G -XX:+HeapDumpOnOutOfMemoryError 
-XX:+UseG1GC -XX:MaxGCPauseMillis=30 
-XX:G1HeapRegionSize=16m 
-XX:InitiatingHeapOccupancyPercent=75 
-XX:+ParallelRefProcEnabled 
-XX:+PerfDisableSharedMem 
-XX:+OptimizeStringConcat 
-Djava.net.preferIPv4Stack=true 
-Djava.net.preferIPv6Addresses=false"))
```
You may have to tweak the heap memory settings (`-Xms -Xmx`), depending on your test requirements.

## Gatling Simulation setup <a name="gatling-sim-setup"></a>
### GatlingRunner <a name="gatling-runner"></a>
{% gist https://gist.github.com/imavroukakis/99289df78baa7f27e93af8d5e5fca0d4 file=GatlingRunner.scala %}

We have to have some way of passing in command-line options, so that we can easily tweak our users-per-second and our test duration. There are several ways to do this, but a good option is [Scallop](https://github.com/scallop/scallop). This gives us (amongst others) GNU-style long option names e.g.
```
--users-per-second=40 --test-duration 20_seconds
```
We default users-per-second to 5 at line 11 and the test duration to 60 seconds at line 12.
```
val usersPerSecond: ScallopOption[Int] = opt[Int](default = Some(5))
val testDuration: ScallopOption[String] = opt[String](default = Some("60_seconds"))
```

At line 29, we check if we're running in `--report-only` mode. This is our stats collection mode, which happens at the end of our run.
```
if (conf.reportOnly.isDefined) {
   props.reportsOnly(conf.reportOnly())
}
```
If that option is not present, we prepare our Simulation by asking Gatling to store our results in a folder with the system's current date/time.
```
else {
  val now = Calendar.getInstance().getTime
  val dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH_mm_ss")          
  props.resultsDirectory(s"results/${dateFormat.format(now)}")
}
```
Finally, at line 36, we kick off our simulation.

### LoadSimulation <a name="load-simulation"></a>

{% gist https://gist.github.com/imavroukakis/99289df78baa7f27e93af8d5e5fca0d4 file=LoadSimulation.scala %}

This is the basic Gatling Simulation. Through the command line parameters supplied, at line 27 we set up our users-per-second and duration and execute our scenario. Our test scenario will perform a  `GET` against _jsonplaceholder_.

## Jenkins Pipeline setup <a name="jenkins-pipeline"></a>
### Jenkinsfile <a name="jenkinsfile"></a>
{% gist https://gist.github.com/imavroukakis/99289df78baa7f27e93af8d5e5fca0d4 file=Jenkinsfile %}

Lines 1-6 should be changed to match your setup so change the following to:

your GitHub repo
```
def gitUrl = 'https://github.com/your_repo/gatling-scale-out'
```

your stored Jenkins Github credentials ID
```
def gitCredentials = 'Github'
```

the number of parallel tests you require
```
def numberOfTestNodes = 5
```

the high-watermark users-per-second above which, your tests will be run in parallel
```
def splitTestsAbove = 50.0
```

your Jenkins JDK installation ID
```
def jdkTool = 'openjdk-11'
```

your SBT installation ID
```
def sbtTool = '1.3.8'
```

A brief explanation of what the pipeline blocks do follows:

#### `environment`
Sets up SBT to be on the common path for all nodes

#### `parameters`
Provides drop-down menus for users-per-second and duration counts.

#### `stage('Checkout')`
Checks out the project from GitHub - works with both Multibranch Pipeline jobs and ad-hoc Pipeline Jobs

#### `stage('Build')`
Builds the projects, packages it and stashes the application for further use in the pipeline

#### `stage('Load Test')`
Looks at the value of `usersPerSecond` and splits the test if the number is equal to, or above the value of `splitTestsAbove`

#### `stage('Collect results')`
Collects all instances of `simulation.log` across the nodes, collates them by running the application in reporting mode and archives the results for download.



### Jenkins Job setup <a name="jenkins-job"></a>

Once you've set up the `Jenkinsfile` to your satisfaction, and committed it to the repo create a new Jenkins _MultiBranch Pipeline_ and configure it as follows:
![MultiBranch Setup](https://dev-to-uploads.s3.amazonaws.com/i/fsszu6ywvk8opt9tyffm.png)
![MultiBranch Setup Jenkinsfile](https://dev-to-uploads.s3.amazonaws.com/i/z7qpiyyxu636we8fkmje.png)

Save the job and Jenkins will scan your repo, and execute an initial run of the job.
![First Run Result](https://dev-to-uploads.s3.amazonaws.com/i/rigejh57syf6w9sxtvfc.png)

**Jenkins may fail the first run of the pipeline, if `Math.round` is not sandboxed**. If that is the case, you should permit this by going to _Manage Jenkins -> In-Process Script Approval_ and approving 
`staticMethod java.lang.Math round double`

After this is done, the `Build with Parameters` option will be available on the left-hand side menu. Clicking on it should bring you to this screen:

![Job Build Options](https://dev-to-uploads.s3.amazonaws.com/i/aun9v70ne3aigb29bac4.png)

Choose 60 users, click on the _Build_ button and wait for the job to finish and then, look at the Build Result.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/cq1line20a5n0i37uoty.png)

Click on the tar archive under _Build Artifacts_ to download the Gatling bundle. Decompress the bundle and open the `index.html` file.

![Gatling Results](https://dev-to-uploads.s3.amazonaws.com/i/uvuemcmo66uqlamgjdrr.png)

In the above example, 3600 requests were executed across 5 nodes for 1 minute. This translates to around 12 users per node. You can confirm this by looking at the build logs.

![Build Logs](https://dev-to-uploads.s3.amazonaws.com/i/j6wsfk9m1zotsm62zqpd.png)

## Conclusion
Congratulations, you now have a basic scaled-out Gatling test. I hope this tutorial has been helpful, but if there's anything that is unclear or puzzling, feel free to drop a comment and I'll do my best to help out.