Keep your AWS CLI config fresh with Cog
Tags: aws, cli, python
Published: 2024-03-28 18:34:04

---
title: Keep your AWS CLI config fresh with Cog
published: true
description: If you have access to a large and shifting set of AWS accounts, it's helpful to dynamically regenerate your AWS CLI configuration file.
tags: aws, cli, python
---

- [The Problem](#the-problem)
- [An Opinionated Solution](#an-opinionated-solution)
- [How I Use It](#how-i-use-it)
  - [Install Cog and aws-sso-config-builder](#install-cog-and-aws-sso-config-builder)
  - [Update the AWS CLI config file](#update-the-aws-cli-config-file)
  - [Refresh my config file](#refresh-my-config-file)
- [Should _you_ use this?](#should-you-use-this)

## The Problem

I have...

- Access to an ever-shifting set of AWS accounts and permissions sets, courtesy of [IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html) (formerly AWS SSO).
- Undying fondness for [aws-vault](https://github.com/99designs/aws-vault/) to securely cache my session credentials.

I want...

- A set of AWS CLI profiles automatically defined in `~/.aws/config`, based on all of the permission sets available to me through IAM Identity Center.
- Those profiles to "just work", in that:
  - They transparently refresh short-lived credentials or launch a browser-based SSO login window as needed.
  - They don't trip up external utilities that read an AWS CLI configuration.
- The ability to add profiles manually for special cases, without interfering with the autogenerated profiles.


## An Opinionated Solution

For a long time, I solved this problem with custom scripts. And that was fine! My workflow felt so specific to my tastes that it wasn't worth packaging.

But then a couple things happened:

- I saw some of [Cog's](https://nedbatchelder.com/code/cog/) use cases and realized my custom scripts were just a low budget reimplementation of what Cog already did well.
- I was exploring different Python packaging tools, but none of my previous work or personal experience gave me a reason to use [Hatch](https://hatch.pypa.io/).

So I decided to convert my custom nonsense into a Hatch-based package that would play nicely with Cog:

https://pypi.org/project/aws-sso-config-builder/

Bird.
Bird.
Stone.

## How I Use It

The aws-sso-config-builder tool doesn't _require_ this workflow - the [usage section](https://github.com/ajkerrigan/aws-sso-config-builder?tab=readme-ov-file#usage) of the README includes alternatives. But this is how _I_ work:

### Install Cog and aws-sso-config-builder

Use [pipx](https://pipx.pypa.io/) to install Cog and my aws-sso-config-builder tool in the same environment:

```console
pipx install --verbose cogapp
pipx inject --verbose cogapp aws-sso-config-builder
```

### Update the AWS CLI config file

Add a block like this to the `~/.aws/config` file:

```
# [[[cog
# import cog
# from aws_sso_config_builder.gen_config import generate_config_blocks
#
# cog.outl(generate_config_blocks(
#     sso_directories=["home", "work"],
#     profile_template="""
#         [profile {profile_name}-sso]
#         sso_session = {sso_session}
#         sso_account_id = {account_id}
#         sso_role_name = {role_name}
#         output = json
#         region = us-east-2
#
#         [profile {profile_name}]
#         credential_process = {aws_vault_path} exec --json {profile_name}-sso
#         output = json
#         region = us-east-2
#     """,
#     regex_replacements={
#         "Production": "prod",
#         "Sandbox": "sbx"
#     },
#     aws_vault_path="/home/aj/go/bin/aws-vault",
# ))
# ]]]
# [[[end]]]
```

Which defines:

- IAM Identity Center directory names / organizations that I have access to
- The skeleton of a config block that should be generated for each account/permission set
- A few regular expressions to customize / normalize profile names
- The path to my aws-vault binary

### Refresh my config file

Now I have:

- Cog available
- aws-sso-config-builder providing a mechanism for looking up permission sets
- A template block in my config file

So when I run:

```console
cog -r ~/.aws/config
```

It replaces everything between `]]]` and `[[[end]]]` with one rendered `profile_template` block per permission set:

```ini
...template block from above...
]]]
...many other profiles...

[profile sbx-AJ-AWSReadOnlyAccess-sso]
sso_session = stacklet
sso_account_id = 111111111111
sso_role_name = AWSReadOnlyAccess
output = json
region = us-east-2

[profile sbx-AJ-AWSReadOnlyAccess]
credential_process = /home/aj/go/bin/aws-vault exec --json sbx-AJ-AWSReadOnlyAccess-sso
output = json
region = us-east-2

[profile sbx-AJ-AWSAdministratorAccess-sso]
sso_session = stacklet
sso_account_id = 111111111111
sso_role_name = AWSAdministratorAccess
output = json
region = us-east-2

[profile sbx-AJ-AWSAdministratorAccess]
credential_process = /home/aj/go/bin/aws-vault exec --json sbx-AJ-AWSAdministratorAccess-sso
output = json
region = us-east-2
[[[end]]]
```

And from there, it's easy for me to switch among any CLI profiles I have. The profile names are verbose enough that it's very clear which account/permissions I have, and through some [shell helpers](https://github.com/ajkerrigan/aws-sso-config-builder?tab=readme-ov-file#fish-convenience-functions) it's still friendly to bounce around:

```console
❯ asp sbx-AJ-AWSReadOnlyAccess

~ via 🐍v3.12.2 (cogapp) on ☁️  sbx-AJ-AWSReadOnlyAccess (us-east-2)
❯
```

## Should _you_ use this?

Probably not. But if you got this far in the post... maybe? Thanks for reading in any case, you deserve some [kanelbullar](https://www.gimmesomeoven.com/swedish-cinnamon-buns-kanelbullar/). Boy those things are delicious.