DevSecOps with AWS- Ephemeral Environments – Creating test Environments On-Demand - Part 1
Tags: aws, iac, devsecops, devops
Published: 2023-10-15 18:01:20

**Level 200 - Intermediate** :eight_spoked_asterisk:

Nowadays, the velocity of application development process increases rapidly due to the adoption and maturity of disciplines and practices like Platform engineering, DevSecOps, GitOps,Caos Engineering, AIOps, FinOps, Generative AI and others. Seeing AI create apps in seconds, resolve issues and respond to incidents is each time more frequent and tomorrow will be normal as a cloud some years ago. However, builders still have many challenges around the security and governance, the technical debt, focus on business case, learn, and innovate faster and adopt those practices and technologies correctly.  

Using the IaC at scale enables practices to apply key pillars of Operational excellence, cost optimization, sustainability, and security. **In this series you can learn how apply an Enterprise CI/CD best practice and the design principles according to the [Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/framework/general-design-principles.html),create Test Environments On-demand, using **Azure DevOps, AWS, and CDK** **. 
Having a single staging environment before production is a common practice, however, is a big disadvantage because it means that developer must either test all their features at once or they must enter a queue, to solve this challenge some companies enable multiple and permanent staging environments for multiple squats for testing in parallel and recurring to some practices as scheduling environments, statics environments adding extra maintenance effort to the team responsible for test environments, increasing the costs due the idle time because those environments are not used all time.

## The challenge

Imagine that your company is modernizing the core business application applying practices such DDD and microservices in AWS cloud, some microservices uses synchronous communication and others asynchronous communication, the developer team require test their features for respective services, but these services also depend on others and no all teams deploy features with the same velocity. Currently, the DevSecOps teams enable custom environments for testing with schedule task that recreate the environments every day, every morning for each team. The company needs to minimize the infrastructure costs but also reduce the operational overload and increase security.  The main AWS services to support the new solution are ECS, VPC, ALB, RDS.

So, how can you solve this problem? :neutral_face:
 

> Read [DevSecOps with AWS- IaC at scale - Getting started](https://dev.to/aws-builders/devsecops-with-aws-iac-at-scale-getting-started-2djh )

First, consider an operation **model based on Decentralized DevOps**, so the Central Platform team must codify infrastructure standards and best practices with reusable modules or packages. 
Second, publish the packages or modules to the private module registry. Finally, the developer team use the packages using interfaces and self-services capabilities such as CLIs, API or UI.

Now, according to the definition of an ephemeral environment: 

_“A temporary, isolated, and replicable version of a software application that is used for testing, previewing, or collaborating on new features or changes. Ephemeral environments are created on demand and discarded after use, saving time and resources. Ephemeral environments can improve the software development cycle by accelerating the feedback loop, enhancing the quality and security of the software, and reducing the bottlenecks and conflicts in the staging environment.”_


Instead of having a predefined number of static and permanent environments, **your pipeline workflow each time a Pull Request is created by a developer must provisioning a new test environment with contents of that Pull Request**. In other words, **use ephemeral environments to get dynamic tests environments**. In this way each developer can test in isolation without any conflicts with what other developers are doing, you pay for the resources of test environments only while you use them, since the test environments are discarded at the end there is nothing to maintain or clean up.

> Note: you can combine both schedule options and dynamic environments.

The diagram below summarizes the scenario.

![Ephemeral Environments](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bgvglqi67r06k0bol1dx.png)

### Scenarios

There are many scenarios for this use case:
•	There is one unique test account for all squats. 
•	Each squat has a test account.
•	Each developer has a test account.
•	There is a central account with core resources such as databases and one of the previous scenarios.

## Hands On - There is one unique test account for all squats

### Overview architecture

The following image depicts the multi-availability zone deployment in an test environment, a DevSecOps CI/CD supported by Azure DevOps, and shared Account for ECR images.


![Overview architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2jku7h20qzlgil087rv7.png)

### Key points

•	The components will be grouped into core environment (VPC, ALB, ECS) and services environment (services definitions, listeners, and rules).
•	Each pipeline could deploy one or more services in the same account for reusing the ALB.
•	Each group of services use a specific range of ports for
•	CDK is the IaC tool, and all stacks and constructs will be written using typescript.
•	The CI/CD system just orchestrated the deployment but the dependencies for between IaC layers must keep with the native way for the IaC Tools.
•	Consider that your developers would like to test all processes locally.


### Requirements 
•	CDK >= 2.94.0
•	AWS CLI >= 2.7.0
•	checkov >= 2.1.229
•	node >= v18.17.1
•	Visual Studio Code
•	Docker >= 24.0.6

### AWS Services

•	[Amazon Elastic Container Service (Amazon ECS)](https://aws.amazon.com/ecs/?nc2=type_a) is a fully managed container orchestration service that helps you easily deploy, manage, and scale containerized applications.
•	[AWS Fargate](https://aws.amazon.com/fargate/?nc2=type_a) is a serverless, pay-as-you-go compute engine that lets you focus on building applications without managing servers. AWS Fargate is compatible with both Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Kubernetes Service (Amazon EKS). Select any OCI-compliant container image, define memory and compute resources, and run the container with serverless compute.
•	[Amazon Virtual Private Cloud (Amazon VPC)](https://aws.amazon.com/vpc/?nc2=type_a) gives you full control over your virtual networking environment, including resource placement, connectivity, and security.
•	[Amazon CloudWatch](https://aws.amazon.com/cloudwatch/?nc2=type_a) collects and visualizes real-time logs, metrics, and event data in automated dashboards to streamline your infrastructure and application maintenance.
•	[Elastic Load Balancing (ELB)](https://aws.amazon.com/elasticloadbalancing/): automatically distributes incoming application traffic across multiple targets and virtual appliances in one or more Availability Zones (AZs).

### The environment setup

> The environment setup for integrate Azure DevOps with AWS and CDK deployments is the same configuration present in the blog [DevSecOps with AWS- Integrate Azure DevOps for CDK deployments Part-1](https://dev.to/aws-builders/devsecops-with-aws-integrate-azure-devops-for-cdk-deployments-part-1-c4e)

You can find the core environment code here: 

{% embed https://github.com/velez94/cdkv2_ephemeral_environment_core %}

The first step is to create a stack for core services, suppose that the core will be used every day from Monday to Friday between office hours, the project structure is:

```bash
.
├── README.md
├── azure-pipelines.yml
├── bin
├── cdk.context.json
├── cdk.json
├── cdk.out
├── diagram.dot
├── docs
├── environment-properties.json
├── environment.d.ts
├── jest.config.js
├── lib
├── node_modules
├── package-lock.json
├── package.json
├── test
└── tsconfig.json

6 directories, 11 files
```

The file `environment-properties.json` defines the properties for VPC, load balancer and ECS Cluster.

```json
{
  "environment": {
    "name": "cdk-ecs-env-demo",
    "inputs": {
      "env": "dev",
      "vpc_cidr_block": "10.0.0.0/16",
      "nat_provider": "instance",
      "nat_gateways": 1,
      "ec2_capacity": false,
      "ec2_instance_type": "t3.medium",
      "allow_ecs_exec": true,
      "enhanced_cluster_monitoring": true,
      "service_discovery_namespace": "demo.svc",
      "load_balanced": true,
      "load_balanced_public": true
      
    }
  }
}

```

For other hand, there are one stack and one construct: `cdkv2_ephemeral_environment_core-stack.ts` and `cdkv2_ephemeral_environment_alb_construct.ts`. 
The outputs are exported via Cloudformation, for example,

```typescript
...
 new CfnOutput(this, "LBDNSName", {
      value: this.loadBalancer.loadBalancerDnsName,
      exportName: `LBDNSName-${props.stackName}`,
    });

    new CfnOutput(this, "ARNALB", {
      value: this.loadBalancer.loadBalancerArn,
      exportName: `ARNALB-${props.stackName}`,
    });

    new CfnOutput(this, "SGALB", {
      value: this.lbSecGrp.securityGroupId,
      exportName: `SGALB-${props.stackName}`,
    });

...
```

This creates a lose coupling between layers and enables the resource utilization for other stacks.

### The pipeline

{% embed https://github.com/velez94/cdkv2_pipelines_azure_devops %}

> Please visit [DevSecOps with AWS- Integrate Azure DevOps for CDK deployments Part-1](https://dev.to/aws-builders/devsecops-with-aws-integrate-azure-devops-for-cdk-deployments-part-1-c4e) and [DevSecOps with AWS- Integrate Azure DevOps for CDK deployments Part-2](https://dev.to/aws-builders/devsecops-with-aws-integrate-azure-devops-for-cdk-deployments-part-2-33i5)

For this example, Azure DevOps is the CI/CD tool and based on previous post the simple configuration is: 

```yaml
# File: azure-pipelines.yml
variables: 
- group: 'ephemeral_envrionments_demo'


resources:
  repositories:
    - repository: cdk_pipelines
      type: git
      name: DevSecOps/cdk_pipelines
trigger:
- master

pool:
  vmImage: ubuntu-latest


stages:
- template: templates/ci_cd.yaml@cdk_pipelines  # Template reference

  parameters:
    Project: ephemeral_env_core
    Language: typescript
    Action: deploy
    ServiceConnection: EphemeralEnvDeployment
    
```
The following image depicts the pipeline execution for deploying core resources:

![The pipeline execution](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qrgtwgoj9ox49423eoe7.png)


Thanks for reading and sharing! :blush: