How Do We Deploy Temporary Acceptance Systems on AWS ?
Tags: legacy, docker, aws, sapbusinessobjects
Published: 2020-04-13 01:41:04

---
title: How Do We Deploy Temporary Acceptance Systems on AWS ?
published: true
date: 2019-12-04 20:35:04 UTC
tags: legacy,docker,aws,sap-businessobjects
canonical_url: 
---

### How Do We Deploy Temporary Acceptance Systems on AWS ?

We want to deliver our products as temporary acceptance systems to be tested by our developers, testers, and pre-sales engineers internally before their release. How do we have setup such system using [CodeBuild](https://aws.amazon.com/codebuild/), [CloudFormation](https://aws.amazon.com/cloudformation) and [ECS](https://aws.amazon.com/ecs/) ?

### Introduction

Our company, GBANDSMITH edits multiple software solutions (the [360Suite](https://360suite.io) portfolio) to help our customers use efficiently [SAP BusinessObjects](https://www.sap.com/products/bi-platform.html) (BOBJ in this article). But as our testing processing are improving, we are increasingly struggling with the installation and deployment of multiple products on test environments. It takes time and generates frustration for developers and pre-sales engineers.

We now have dedicated resources for testing itself, and we don’t want them to struggle with the installation/provisioning phases but to focus on testing.

![](https://cdn-images-1.medium.com/max/1024/1*kGylXd-BVfFqoFKnhHWDmw.png)<figcaption><a href="https://undraw.co/illustrations">unDraw</a> illustration</figcaption>

The **GB&SMITH Labs** team has been asked to find a solution on this issue!

We have worked to deliver a global solution that answers the following needs:

- developers and pre-sales engineers can **create themselves** a solution
- the solution will **contain our 2 main products** ([360WebPlatform](https://360suite.io/sap-business-objects/security/) — _named 360wp in schema_, and [360Eyes](https://360suite.io/sap-business-objects/audit-impact-analysis-metadata/)), and optionally a BOBJ engine
- it must be possible to select any _git branch_ ( **work in progress** ) or **released**  **version** (available as _git tag_) of the 2 products
- the solution must be easily available, like an accessible URL endpoint
- multiple solutions can be created at the same time and used by multiple persons

### The design

From the beginning, we wanted to work on this project only with resources available on our main cloud environment, **Amazon Web Services**. The goal was to prove that the team is able to deliver a real cloud solution, and to trial-and-error on some new tools available on AWS. So the global design of the solution is intended to use instances only if needed.

![](https://cdn-images-1.medium.com/max/1024/1*ZA9NGBl1sUEVkGqqSuV6LA.png)

We are using the following products of Amazon Web Service:

- **CodeBuild** is the Continous Integration / Continous Deployment tooling that triggers the solution creation. It is used directly by our teams.
- The solution is mainly deployed on **Elastic Container Service** (ECS) that is a container orchestration solution. Our products are deployed as docker containers.
- We rely on **Elastic Container Repository** (ECR) to store our private docker images.
- We rely on **Amazon Relational Database Service** (RDS) to host our PostgreSQL database instance, used by both 360Eyes and 360WebPlatform. The EC2 BOBJ instance is also able to use it.
- We have a single **EC2** (Elastic Cloud Compute) instance to host our BOBJ system.
- The creation of the system (ECS + EC2 + RDS) is orchestrated by a **CloudFormation** script, that helps us bring up and down the full stack.

### The Stack

Let’s dig a bit on our choice. Choosing the right tools is not obvious, so let me take a few minutes to explain why we have created such a stack.

#### Why use ECS ?

ECS is one of the solutions in AWS if you want to deploy docker containers. We think that it is easier to deal with docker containers and images to deploy our products. It is easier and faster than working with instance images:

- Build the code using our classic Maven tooling ( maven clean install )
- Create a docker image ( docker build . )
- Tag and push the image to ECR ( docker tag ...and then docker push ...)
- Run the ECS task based on ECR

We are dealing here with couple of Mb (around 500 Mb for a docker image) instead of multiple Gb for a instance image. There is no easy way to generate a instance image containing our product from a CI solution like **CodeBuild** (the best I know in this area is the very good [Packer](https://www.packer.io) tool from HashiCorp).

#### Why use an EC2 instance ?

In an ideal scenario, I would avoid an EC2 instance. It relies on an EC2 image (Amazon Machine Image), and it is bulky to modify. But here we want to deliver a BI solution, and officially, this solution is not compatible with Docker containers. I know some users are trying to dockerize BOBJ ([here](https://blogs.sap.com/2017/02/15/sap-bi-and-docker-installing-and-scaling-sap-businessobjects-with-docker/) for example), but we haven’t succeed in this task.

#### Why use RDS ?

RDS is the de-facto standard solution to deploy a database in AWS. Both our products relies on a database, so we need to ship one. We chose RDS.

Actually it is not the only solution, we can also imagine adding a database container in our ECS task, but this solution complexifies the connectivity between the BOBJ instance and the database.

#### Why use CloudFormation ?

CloudFormation is the _Infrastructure as Code_ solution in AWS. There are also other tools available like Terraform from [HashiCorp](https://www.terraform.io) (again!).

We chose to use CloudFormation in this project because it allows us to create a full stack and also to **tear-down the stack** very easily from the AWS API when the user doesn’t need the solution anymore (it is a bit more difficult to teardown a terraform setup, not a single AWS API call !)

#### Why use CodeBuild ?

Our company mainly uses [Jenkins](https://jenkins.io) as CI solution. Starting on CodeBuild is not obvious ! The goal of the project was to use a completly AWS solution without any external product.

CodeBuild is quite a mature product now, which allows to setup a project with conditional builds, and to consume multiple git repositories. It also allows to push docker images to ECR, and to trigger the CloudFormation stack creation. Here is an exemple of the _buildspec_ provided in the project:

```
version: 0.2

cache:
 paths:
 - '/root/.m2/\*\*/\*'

phases:
 install:
 runtime-versions:
 java: openjdk8
 pre\_build:
 commands:
 - echo Logging in to Amazon ECR...
 - $(aws ecr get-login --no-include-email --region $AWS\_REGION)
 build:
 commands:
 - |
 if test "$BUILD\_EYES" = "1"; then
 echo "Build 360eyes docker image and push it to ECR";
 cd $CODEBUILD\_SRC\_DIR/360ace/eyes;
 mvn clean install -Pdocker;
 docker tag 360eyes/360eyes.rest.service:latest xxx.dkr.ecr.$AWS\_REGION.amazonaws.com/360ace/eyes:latest;
 docker push xxx.dkr.ecr.$AWS\_REGION.amazonaws.com/360ace/eyes:latest;
 fi
 - echo Launching CloudFormation template
 - cd $CODEBUILD\_SRC\_DIR/360ace
 - aws cloudformation create-stack --template-body file://360ace.yaml --stack-name ace-$(date +%d%H%M)
```

This is just does the following:

- build the product and docker image (using this [maven plugin](https://github.com/spotify/dockerfile-maven))
- tag and push on **ECR** the docker image
- start the **CloudFormation** stack

A good point is that it allows to checkout a specific branch (like a developer working branch) or a specific tag (a release) of our product repositories.

### Managing resources on AWS

Here we see that we have some temporary resources created when a solution is up, and some more stable resources that continue to live even when no solution is running:

- **Temporary** resources: EC2 instance, ECS service, RDS database
- **Stable** resources: ECR repositories, CodeBuild project

The **temporary** resources are directly managed by the CloudFormation stack: when the stack is created it fires up all the linked resources. When the stack is stopped, then the linked resources are deleted.

The **stable** resources are designed to live all the time. But instead of creating them directly in the AWS Console, we prefer to use **Terraform** (again again !) to manage that. It allows us to have a clean setup that we can reproduce on multiple AWS accounts. We are also able to review the infrastructure changes within a git repository and to take full advantage of _Infrastructure as Code_.

![](https://cdn-images-1.medium.com/max/1024/1*zWhHZUDWuxJwfEoumC1-OQ.png)<figcaption><a href="https://undraw.co/illustrations">unDraw</a> illustration</figcaption>

### End result

As a result now our employees can start a temporary solution of our solutions just by triggering a CI job:

![](https://cdn-images-1.medium.com/max/860/1*U8ZB-aFdQJnzLgy0IpLN6Q.png)

And all is presented in a _quick and dirty_ React application (the home container):

![](https://cdn-images-1.medium.com/max/891/1*ZDnLCm9a-7EdsJXSZlW3Aw.png)

### Next steps

OK, the solution is here and available but there is still a lot of room for improvements.

#### How to trigger the solution creation ?

Because we are only on AWS, we imagined some additional improvements to our system: it is possible to setup a _chatbot_ that is able to _create_ and _tear down_ a system just by speaking to it. It communicates only with the AWS API (CodeBuild and CloudFormation):

![](https://cdn-images-1.medium.com/max/825/1*qjlDKlpH9ZyLtOiAoVwHvQ.gif)

#### How to speed up the stack creation ?

Currently we are relying on a RDS snapshot to pop-up some existing data in the RDS database. It is easy to setup but it is [dramatically long to fire](https://stackoverflow.com/questions/35490807/rds-snapshot-restore-taking-too-long), around 20 minutes…We are thinking on how we can speed up this (maybe by using a different system).

#### Add more flexibility

We can also imagine adding an option to select a specific BOBJ version to start (BI4.1, BI4.2, …), or select the RDS backend type (MySQL, PostgresSQL, Oracle…). All of that is possible and we are waiting for our internal customers to get some usage feedback !

### To Conclude

Cloud is hard. We have just scratched the surface of the issues and the options offered by a cloud platform like **AWS**. We still have lots of methods, skills, and tools to learn. But we are very happy to be able to deliver a working tool to our co-workers, waiting for the next project !

* * *