Sophisticated use-case of AWS CDK that leverages AWS SDK
Tags: devops, cdk, awssdk, enterprise
Published: 2023-11-01 02:06:36

by Udaybhaskar Sarma Seetamraju (ToSarma gmail)
Oct 16 2023

## One sentence summary
A real-world sophisticated use-case exists for leveraging AWS SDK within AWS CDK code, when deploying cloud-native Non-Kubernetes solutions, within Enterprise-environments where IP-address space is a major constraint.

## One Paragraph Summary
For those of us, who do Not work for the leading-edge tech-companies (which is the overwhelming majority of the Fortune 1000), private IP addresses are a fact. That is, we face serious constraints in how many VPCs and AWS Accounts can exist. Also, frequently due to lack of an [IPAM system](https://efficientip.com/glossary/what-is-ipam/), the ability to automatically create and dispose AWS Accounts is missing.  In such Enterprise contexts, as shown in this article, using AWS SDK (within AWS CDK code) leads to far simpler and far more robust solutions, specifically when deploying cloud-native Non-Kubernetes solutions. 

## Focus Areas & Keywords
DevOps, Enterprise Environments, “Dirty” VPCs, AWS CDK, AWS SDK, CLI Profiles, “Everything is Disposable”.

# The main article

![HighQuality](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5s22s39qa5uwu281g0fi.png)

## Enterprise Context: A unwavering focus on Production

**_My Philosophical “True North”_**: Regarding AWS Resources, my perspective is: AWS SDK is for _obtaining details on what already exists_, whereas AWS CDK (incl. Cloudformation) is about _what we need to create_.  In other words, I personally do Not use AWS SDK to create new AWS Resources, nor do I “mildly” manipulate any pre-existing AWS Resources.  FWIW: CDK itself relies on SDK.

I belong to the group of people who do Not like Cloudformation in which each resource has a “`Condition:`”.  The best practice (of maintainability / supportability) is to have Cloudformation-templates (CFT) that simply & clearly tell you what was deployed; It’s easy to predict/understand what will happen if you re-deploy.  In many industries, Cloudformation-templates, Stack-drifts and Change-Sets are like god’s gifts, but that is out of scope for this article.

**_“Corporate Life”_**: Commonly enough within Enterprise environments, either there is a limited ability, or there is an outright ban on the ability to create “fresh new” VPCs and/or AWS Accounts.  An even more important nuance is, this limitation/absence can be quite severe within Production environment/VPCs, even as developers may experience “freedom” within sandbox/PoC/Development/Integration environments.

Per the best practice for designing solutions, we must have all identical environments. So, developers should “copy/simulate” the _constraints of Production_ into all the Non-prod environments.

**_Use-Case / Scenario_**: This article is specific to one specific use-case, where CDK-based deployment is done manually (via “`cdk`” CLI).  Personally, I experience this use-case early in every project, when the deployment-architecture diagram (onto AWS) is constantly in flux.  The iterative approach using the CI/CD pipeline to improve your CDK is too horribly slow, un-acceptably inefficient and error-prone.

For any of my projects, for 3 specific environments (namely _sandbox_, _PoC_ and _development_ environments), I rely on **<u>manual</u>** CDK-CLI based deployments. 

## The core security needs


![SecurityHazard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n8w98pgyg4khknu1xrlu.png)

-  CDK offers a great advantage within DevSecOps pipelines, something we should all take FULL advantage of. Via CDK-code, we can _very easily ensure all_ IAM Roles and Network-security components are also covered robustly, for each component of overall solution.
- The example in this article focuses solely on AWS Security Groups (**<u>SGs</u>**), since use-cases to **<u>share at least one SG</u>** (across various components of a solution) is a common enough.  More details on this is throughout this article. Note: Never share IAM Roles across components of a solution.
- In the early release of a medium/large application, it is very common to have an “**Application-wide**” **Security Group** (“_AppWide-SG_”).  This is a good best-practice to remember, as it forces developers to start day-one, in strictly using SGs to limit access to every single component.

## What is a Dirty VPC?


![DirtyWorkArea](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p6mm63qrrm5wqbtdw1ea.jpg)

- The concept of a “Dirty VPC” is simply an implication of the Enterprise constraint that “_One is restricted in creating new VPCs within DevSecOps pipelines_”
- The dictionary word “_Dirty_” has a negative meaning.  But, for this article, we will interpret that word to mean that the VPC already has one of more desired AWS resources; in this example, Security-Group(s).
- I love the philosophical approach off "_Everything is Disposable_". I practice it a lot, including the use of cloud IDEs. But, “Dirty VPCs” are the proverbial “_immovable object_” within Enterprise environments.

## Some foundational facts


![OldManuscript](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8txle0xgs5of2jp75kxm.png)

- FYI: Per [AWS Official documentation](https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/guide_credentials.html#default-credential-chain), AWS SDK authenticates (a.k.a. _**AuthT**_) based on one of the following 3 mechanisms:
    1. `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` environment variables.
    1. Based on “`--profile`” CLI arguments.
    1. Based on “`default`” AWS Profile in `~/.credentials` file.
- FYI: Similarly, [AWS CDK relies on](https://docs.aws.amazon.com/cdk/v2/guide/environments.html) the above 3 for _AuthT_ a.k.a. _authen**T**ication_.
- FYI: Similarly, re: the “`Region`”, both AWS CDK & AWS SDK determine it either based on environment variables, or “`--region`” CLI-arg or on the region specified within “`default`” AWS Profile in `~/.credentials` file.
- _Note_: In an Enterprise work-environment, due to pervasive use of `SSO` (Single-SignOn), you are highly likely to be using the “`--profile`” cli-arg.

## DRY (Don’t Repeat Yourself) Problem


![DRY](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ywp2thts6rmon6ygnvoh.png)

- _<u>Best Practice</u>_: For _sandbox/PoC/development_ environments, exclusively for **<u>manual</u>** deployments, CDK best practices suggest that you use “`--profile`” CLI-arg (as well as the “`—region`” CLI-arg. In contrast, for **all other** environments, especially within DevSecOps pipelines, you should be _exclusively rely_ on `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` environment variables.  CDK’s & SDK’s best-practices suggests the use of “`--profile`” CLI-arg and the “`--Region`” CLI-arg (instead of using “`default`” AWS Profile). 
- <u>_Limitation_</u>: Per the AWS Support-case 13701110271, dated Sept 2023, AWS confirmed that CDK uses AWS SDK underneath, but, unfortunately, we can Not get “access” to the SDK. That implies that our code **<u>must explicitly initialize AWS SDK, by providing it</u>** the credentials.
- <u>_The Issue_</u>: the CDK will Not share details on how it successfully-authenticated -- whether via Env-Vars, or via the “`--profile`” CLI-arg, or via the “`default`” AWS Profile.
- **<u>_The PROBLEM_</u>**: We have a “[DRY problem](https://en.wikipedia.org/wiki/Don't_repeat_yourself)”, when we use “`--profile`” CLI-arg and the “`--Region`” CLI-arg (instead of “`default`” AWS Profile); AWS CDK gets that information, but Not the AWS SDK.   So, we need to “copy/clone” that information and send it to AWS SDK too (for it to _AuthT_ too). See code-snippet #2 below, for the implementation.

## The official Workaround: AWS-SDK AuthT


![SnailOnRocket](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ad1po6x13gvl2zy09tl.png)

Within my CDK code, I mandate 2 additional _CLI-arguments_ as: “`--context AWSProfile=...`” and  “`--context AWSRegion=...`” To enforce these additional _cli-args_, use **code-snippet #1** (see below).
FYI: The above makes your command for manual CDK-deployment look like:-

```
cdk deploy|synth \
     --profile ${AWSPROFILE} \
     --region ${AWSREGION} \
     --context AWSProfile=${AWSPROFILE} \
     --context AWSRegion=${AWSREGION}
```

Per the AWS Support-case 13701110271, dated Sept 2023, AWS FYI: I got **2023-specific** confirmation on what I’m recommending within this article: “_What you are doing with the context file is the best approach to this_.”  Please do Not assume this remains true for 2025 & beyond.

## Code Snippet #1 - “`cdk`” CLI’s mandatory CLI args

Within your CDK project, please copy these lines within the only file: “`./bin/*.ts`”, ideally preceding the lines starting with “`new Stack(app, .. {`”.

```
const myAWSProfile = scope.node.tryGetContext('AWSProfile');

if ( ! AWSProfile ) {
    console.error(`!! FATAL-ERROR: CLI-args are missing:-->      --context AWSProfile=....\n\nExiting ...`);
    process.exit(1);
}

const myAWSRegion = app.node.tryGetContext('AWSRegion');

if ( ! AWSRegion ) {
    console.error(`!! FATAL-ERROR: CLI-args are missing:-->      --context AWSRegion =....\n\nExiting ...`);
    process.exit(1);
}

// OPTIONAL CLI-arg
const isDirtyVPC_cliarg :string | undefined = scope.node.tryGetContext('dirtyVPC');
const isDirtyVPC :boolean = (isDirtyVPC_cliarg != undefined) && (isDirtyVPC_cliarg !== "false");
if ( isDirtyVPC ) {
    console.log(`Ok! Deploying to a "dirty" VPC that may already have certain SecurityGroups!`);
}
```

## Code Snippet #2 - AWS-SDK: Authenticating into AWS-API

```
import * as EC2 from ‘aws-cdk-lib/aws-ec2';
..
..
  const aws_sdk_credentials = {
      credentials: fromIni({
          profile: myAWSProfile,
      }),
      // region: process.env.CDK_DEPLOY_REGION || process.env.CDK_DEFAULT_REGION,
      region: myAWSRegion,
  };

  const stsclient = new STSClient(aws_sdk_credentials);
  const stscommand = new GetCallerIdentityCommand({});
  const stsresponse = stsclient.send(stscommand).then((data) => {
     const AWSProfile_derived = data.Arn?.split('/')[1] || “UnknownProfile1";
     console.log(`Sanity-Check: AWSProfile in ARN = ${AWSProfile_derived}`);
  }
```

## How to keep your CFT simple & highly maintainable

This article wouldn’t be complete without ready-to-use CDK code-snippets, to conditionally/optionally generate resources within your Cloudformation-Template (CFT), after AWS-SDK calls are used to determine those conditions.

In this example (code snippet #3), we use AWS-SDK calls to determine if an Security-Group exists;  If missing, the creation of the SG (AWS Security Group) is included in the cloudformation generated by SDK.

## Code Snippet #3 - Conditional Cloudformation

```
import * as EC2 from ‘aws-cdk-lib/aws-ec2';
import { EC2Client } from ‘@aws-sdk/client-ec2';
..
..
async function lookupSecurityGroup( securityGroupName: string, 
                                    aws_sdk_credentials: any, )
              : Promise<string | undefined>
{
    const ec2client = new EC2Client( aws_sdk_credentials );
  ..
  .. // see full version of THIS function -> at bottom of this article.
  ..
    const sgresponse = await ec2client.send( sgcommand );
    if ( !sgresponse.SecurityGroups || sgresponse.SecurityGroups!.length < 1) {
        throw .. ..
    } else {
      return new Promise((res) => res( sgresponse.SecurityGroups[0].GroupId );
    }
} // end of async function.
```

// now let’s utilize the above function !!

```
    await lookupSecurityGroup( SGNAME, aws_sdk_credentials )
    .then((sgid) => {
        myAppWideSecurityGroup = EC2.SecurityGroup.fromLookupByName( this,  SGNAME, SGNAME, defaultVpc );
        if ( ! isDirtyVPC ) {
           throw Error( `!! SAFETY-CHECK ERROR !! SG ${SGNAME} already exists.  You did NOT provide CLI-arg "--dirtyVPC".\nExiting with error-code ...` );
        }
     }).catch( (e) => {
        console.log( `Ok. SG Does Not exist. So, creating new SG named ${SGNAME} ...` );
        const mySGProps: EC2.SecurityGroupProps = {
                    vpc: defaultVpc,
                    securityGroupName: SGNAME,
                    description: SGDESC,
                    allowAllOutbound: true,
                    disableInlineRules: true,
        };
        myAppWideSG = new EC2.SecurityGroup( this, SGNAME, mySGProps );
        // commented out: myAppWideSG.addEgressRule( EC2.Peer.anyIpv4(), EC2.Port.allTcp(), "Any Port OUTbound" );

        const cfn_sg = myAppWideSG.node.defaultChild as EC2.CfnSecurityGroup;
        cfn_sg.overrideLogicalId( SGNAME.replace(/-/g, '') ); 
```

The last line of code removes all hyphens from SGNAME, so the resulting string can be used as Resource-Name within Cloudformation-template.

Note: I'm ignoring `addEgressRule` since '`allowAllOutbound`' is set to `true` by default; To add customized rules, set `allowAllOutbound=false`.

# Key Takeways


![RibbonOnFinger](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pblr8c7k24oe1k2b477s.png)

Even though this article focused on a very valid common real-world scenario w.r.t. AWS Security Groups, this article offers a simple clean way to combine AWS-SDK with AWS-CDK, to can design/write CDK code that cleanly checks for a pre-existing component (say, a Fargate ECS-Cluster), and skip the generation of Cloudformation accordingly.

# APPENDIX: full-code

```
import { EC2Client, DescribeVpcsCommand, DescribeSecurityGroupsCommand } from '@aws-sdk/client-ec2';
import { SecurityGroup } from '@aws-sdk/client-ec2';

export async function lookupSecurityGroup( securityGroupName: string,  aws_sdk_credentials: any, ): Promise<string>
{
  const ec2client = new EC2Client( aws_sdk_credentials );
  const filter = {
      Filters: [ { Name: "isDefault",  Values: ["true"] } ],
      MaxResults: 5, // Number("int");    Attention! Minimum value is 5. Maximum value of 1000.
      DryRun: false,
      // VpcIds: [ "vpc-0123456", ],
  };
  const vpccommand = new DescribeVpcsCommand( filter );
  const vpcresponse = await ec2client.send(vpccommand);
  if ( ! vpcresponse.Vpcs || vpcresponse.Vpcs.length < 1 ) {
      throw new Error('No default VPC found');
  }
  const defaultVpcId = vpcresponse.Vpcs![0].VpcId!;
  // DescribeVpcsResult: https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-ec2/interfaces/vpc-8.html
  // {  Vpcs: [{
  //       CidrBlock: "STRING_VALUE",
  //       DhcpOptionsId: "STRING_VALUE",
  //       VpcId: "STRING_VALUE",
  //       IsDefault: true || false,
  //       Tags: [ // TagList
  //       .. ..

  // ---------------------------------- 
  const sgfilter = { // DescribeSecurityGroupsRequest
      Filters: [
          { Name: 'vpc-id', Values: [defaultVpcId], },
          { Name: 'group-name', Values: [securityGroupName], },
               // NOTE: Simpler to use "GroupName" instead !!!
      ],
      DryRun: false,
      MaxResults: 5, // Number("int");    Attention!  Minimum value is 5. Maximum value of 1000.
      // GroupIds: [ "sg-12345678", ],
      // GroupNames: [  "STRING_VALUE", ],
      // NextToken: "STRING_VALUE",
  };
  const sgcommand = new DescribeSecurityGroupsCommand( sgfilter );
  const sgresponse = await ec2client.send(sgcommand);
  // { // DescribeSecurityGroupsResult
  //   SecurityGroups: [ // SecurityGroupList
  //     { // SecurityGroup
  //       Description: "STRING_VALUE",
  //       GroupName: "STRING_VALUE",
  if ( ! sgresponse.SecurityGroups || sgresponse.SecurityGroups!.length < 1 ) {
      throw new Error(`No SG found with name ${securityGroupName}`);
  }

  return sgresponse.SecurityGroups![0].GroupId!;
}
```