Cost-Efficient Integration Testing on vendor appliances running on EC2
Tags: ec2, aws, maven, bi
Published: 2018-11-06 16:02:44

---
title: Cost-Efficient Integration Testing on vendor appliances running on EC2
published: true
date: 2018-11-06 16:02:44 UTC
tags: ec2,aws,maven,bi
canonical_url: https://medium.com/gbandsmith/cost-efficient-integration-testing-on-vendor-appliances-running-on-ec2-7101cf1ba895
cover_image: https://cdn-images-1.medium.com/max/1024/0*HarFT6ZCRY1EQnZR
---

We are building applications that are relying on monolithic commercial applications like [SAP Business Objects](https://www.sap.com/products/bi-platform.html) or [Tableau Software](https://www.tableau.com/). Following our quality standard, we want to run reliable, efficient and cost-effective tests to validate our work. Here is how we are doing that using AWS EC2 instances.

## The need

**Integration Testing** and **Acceptance Testing** are necessary steps to insure quality in our products. But this kind of tests needs to interact with existing vendor solutions, not available in an [SaaS](https://en.wikipedia.org/wiki/Software_as_a_service) model. For example we are working a lot around SAP BusinessObjects and we need to run tests against this kind of software.

![](https://cdn-images-1.medium.com/max/482/1*xTj_39MGz9jOH7dQFi2ukA.png)<figcaption>Integration Tests relying on external systems</figcaption>

## The first idea

An easy solution is to deploy an integration server, secure it to deny everyone the possibility to modify the content (it needs to stay stable since tests are relying on the content), and run it all time. Do that for example in your existing local infrastructure or drop that on a EC2 at AWS. This is a simple solution but it might be very costly (you only need it a couple of minutes a day ! ) and because content need to be the same you probably want to reset it at its original state before performing additional tests. You pay around **143 US$** for a t2.xlarge when it’s up all time.

## The solution

Because our build process is managed by Maven we are searching for a Maven plugin solution for that. And lo and behold it exists !

{% github gbandsmith/ec2-maven-plugin no-readme %}

We have decided to fork an existing project and adapt it to our specific needs. The plugin is able to create an EC2 instance on-demand based on an existing image when the integration test goal is launched and terminates it at the end of the goal. The customization allows us to wait until the vendor application is fully started on the EC2 (by listening to TCP ports).

It doesn’t rely on an existing EC2 instance but on a Amazon Machine Image (AMI) and this is good news :

- each integration test ran is done on the exact same AMI each time, we can rely on the server's content being absolutely consistent through time.
- multiple integration tests can be launched in parallel (2 developers, jenkins server…), each one will create a specific instance.

![](https://cdn-images-1.medium.com/max/592/1*8uBl-TkCckXinRdpu4Hu0w.png)<figcaption>Process of the Maven Plugin actions</figcaption>

The project integration is very simple. You can call for the proper maven plugin in the pom.xml file :

```
<build>
<plugins>
    <plugin>
     <groupId>com.gbs.maven.plugins</groupId>
     <artifactId>ec2-maven-plugin</artifactId>
     <version>1.1.8</version>
     <configuration>
      <accessKey>XXX</accessKey>
      <secretKey>xxxx</secretKey>
      <ami>${bobj.ami.id}</ami>
      <key>gbandsmith</key>
      <type>t2.large</type>
      <securityGroups>
       <securityGroup>bobj</securityGroup>
      </securityGroups>
      <tags>
       <tag>
        <key>Name</key>
        <value>sapbi-integration-test</value>
       </tag>
      </tags>
      <waitPorts>
       <waitPort>6400</waitPort>
       <waitPort>6402</waitPort>
      </waitPorts>
     </configuration>
     <executions>
      <execution>
       <id>start-ec2-instance</id>
       <phase>pre-integration-test</phase>
       <goals>
        <goal>launchinstance</goal>
       </goals>
      </execution>
      <execution>
       <id>stop-ec2-instance</id>
       <phase>post-integration-test</phase>
       <goals>
        <goal>terminateinstance</goal>
       </goals>
       <configuration>
        <instanceId>${ec2.instance.id}</instanceId>
       </configuration>
      </execution>
     </executions>
    </plugin>
</build>
```

The _execution_ and _terminate_ goals are linked to _pre-integration-test_ and _post-integration-test_ phase of the classic [Maven Failsafe plugin](https://maven.apache.org/surefire/maven-failsafe-plugin/).

To use the AWS DNS entry we use the configuration of failsafe to set a _System Property_ variable that we can use in our code:

```
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <configuration>
    <systemPropertyVariables>
      <myvar>${ec2.instance.dns}</myvar>
    </systemPropertyVariables>
  </configuration>
</plugin>
```

We now have a robust integration test system that rely on AWS, with a very low cost (around 30US$ per month for a 10-person project).

## Next

There is still some room for improvement. A few ideas that we have:

- Replace the failsafe plugin by something else. Currently when a test fail then the _post-integration-test_ step is not trigged so the EC2 instance is not terminated.
- Optimize the build when we build a multi-module Maven project (currently each Maven module starts an independent instance and we‘re losing time when multiple modules are running IT tests).