Using undocumented AWS APIs
Tags: aws, cloud, python, iam
Published: 2023-10-22 13:04:13

[TL;DR just give me the code](https://github.com/binxio/aws-iamv2)

While evaluating some existing IAM policies in a codebase, I found myself repeating the same steps over and over again: navigate Google and search `iam actions servicename` and look up information about the actions used.

Pro tip: it is much easier to just bookmark this one: [https://docs.aws.amazon.com/service-authorization/latest/reference/reference_policies_actions-resources-contextkeys.html](https://docs.aws.amazon.com/service-authorization/latest/reference/reference_policies_actions-resources-contextkeys.html)

The information I needed for my policy validation work is quite simple:
- what are all the services?

- what are all the actions a service has?

- what actions match a pattern like "Desc*" for a service?

- what are mandatory and optional resources for each action?

- what condition keys can be used?

- is the action readonly, readwrite or something else?

The first place to go to find out if this information is somehow exposed would be the AWS SDKs. I looked through the boto3 documentation on the `iam` service and came up empty.

After doing a lot of policies using the manual process, I remembered that AWS has a policy-editing tool in the console that seems to be using the information I was looking up manually. So I started my adventure by trying to automate my struggles.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vfweb3j2tbo9ttn3ewrv.png)

I decided to invest some time in the policy editor, which was using some kind of API I could use to automate some things. So with the Chrome Inspect pane using the network tab, I saw a lot of http requests to:

[https://us-east-1.console.aws.amazon.com/iamv2/api/iamv2](https://us-east-1.console.aws.amazon.com/iamv2/api/iamv2)

After some experimentation and fiddling with cookies and CSRF tokens, I found out how this undocumented API worked. So I cooked up a little Python to automate that. Since it is only 80 lines of code, I'll share it here. I will probably make an installable Python package of it soon. The repository with the code and some examples is [https://github.com/binxio/aws-iamv2](https://github.com/binxio/aws-iamv2).

```python
import requests
import json
import boto3
from bs4 import BeautifulSoup

# composePolicy, decomposePolicy, checkMultiMFAStatus, createX509, cuid, generateKeyPairs
methods = { "services"                    : lambda p: None,
            "actions"                     : lambda p: { "serviceName": p, "RegionName": "eu-central-1" },
            "resources"                   : lambda p: None,
            "contextKeys"                 : lambda p: None,
            "globalConditionKeys"         : lambda p: None,
            "getServiceLinkedRoleTemplate": lambda p: { "serviceName": p },
            "policySummary"               : lambda p: { "policyDocument": p },
            "validate"                    : lambda p: { "policy": json.dumps(p), "type": "" } }

class ConsoleSession:
    def __init__(self, boto3_session):
        self._credentials = boto3_session.get_credentials()
        self._signed_in = False
        self._csrf_token = None
        self._cache = {method: {} for method in methods}
        self._rsession = requests.Session()

    def __getattribute__(self, name):
        if name in methods:
            def make_lambda(method, converter):
                return lambda param=None: self.get_api_result(method, converter(param))
            return make_lambda(name, methods[name])
        else:
            return object.__getattribute__(self, name)

    def signin(self):
        token = json.loads(self._rsession.get(
            "https://signin.aws.amazon.com/federation", 
            params={
                "Action": "getSigninToken",
                "Session": json.dumps({
                    "sessionId": self._credentials.access_key,
                    "sessionKey": self._credentials.secret_key,
                    "sessionToken": self._credentials.token
                })
            }
        ).text)["SigninToken"]
        self._rsession.get(
            "https://signin.aws.amazon.com/federation",
            params={
                "Action": "login",
                "Issuer": None,
                "Destination": "https://console.aws.amazon.com/",
                "SigninToken": token
            }
        )
        for m in BeautifulSoup(self._rsession.get(
            "https://us-east-1.console.aws.amazon.com/iamv2/home#",
            params={ "region": "eu-central-1", "state": "hashArgs" }
        ).text, "html.parser").find_all("meta"):
            if m.get("name") == "awsc-csrf-token":
                self._csrf_token = m["content"]
        self._signed_in = True

    def get_api_result(self, path, param=None):
        not self._signed_in and self.signin()
        params = json.dumps(param)
        if self._cache[path].get(params, None):
            return self._cache[path][params]
        self._cache[path][params] = json.loads(self._rsession.post(
            "https://us-east-1.console.aws.amazon.com/iamv2/api/iamv2",
            headers={
                "Content-Type": "application/json",
                "X-CSRF-Token": self._csrf_token,
            },
            data=json.dumps({
                "headers": { "Content-Type": "application/json" },
                "path": f"/prod/{path}",
                "method": "POST",
                "region": "us-east-1",
                "params": {},
                **({ "contentString": params } if params else {})
            })
        ).text)
        return self._cache[path][params]
```

A few things about this code:
- The `ConsoleSession` class takes a `boto3.Session` as input. This session needs no actual rights to AWS.

- The code might look a bit strange because I wanted to make it as a *dynamic class* so I only had to add one line to implement an extra API endpoint. This uses the `__getattribute__` override and the `methods` object.

- I use the `requests` module and start a `requests.Session()` that does much of the heavy lifting handling cookies needed for the http requests to succeed.

- To fetch the `awsc-csrf-token` from the page I use `BeautifulSoup`

- The method in `methods` were not all discover by me, I did a search on `iamv2` on github and found some json files that were already detailing this API.

- Since the information retrieved for the API calls I needed does not change per request I implemented a simple caching feature. For some methods like `policySummary` and `validate` this might not be optimal.

The actual signing in is done using three HTTP requests:

1. Getting a `SigninToken` from `signin.aws.amazon.com/federation`

2. Use the token to `login` to `https://console.aws.amazon.com/`

3. Retrieve `https://us-east-1.console.aws.amazon.com/iamv2/home#` to get the `awsc-csrf-token` from the page.

## Example usage

The code below demonstrates example usage:

```python
import boto3
from iamv2 import ConsoleSession
import re

awssvcs = {}
console_session = None

def get_iam_info():
    global console_session
    boto_session = boto3.Session(region_name='us-east-1')
    console_session = ConsoleSession(boto_session)

    services = console_session.services()
    for service in services:
        name = service["serviceName"]
        if name not in awssvcs:
            awssvcs[name] = { "parts": [] }
        awssvcs[name]["parts"].append(service)

def get_statement_actions(statement):
    result = []
    actions = statement.get("Action") or statement.get("NotAction")
    reverse = "NotAction" in statement
    reverse = not reverse if statement["Effect"] == "Deny" else reverse
    actions = [actions] if isinstance(actions, str) else actions
    for action in actions:
        service, act = action.split(':')
        if "Actions" not in awssvcs[service]:
            awssvcs[service]["Actions"] = console_session.actions(awssvcs[service]["parts"][0]["serviceKeyName"])
        actrgx = act.replace('*', '[A-Za-z]+')
        for svc_action in awssvcs[service]["Actions"]:
            if bool(re.match(actrgx, svc_action["actionName"], flags=re.IGNORECASE)) ^ reverse:
                result.append(svc_action)
    return result

def get_policy_actions(policy):
    for statement in policy["Statement"]:
        yield get_statement_actions(statement)
```

`get_policy_actions` will simply list all the actions allowed by the statements in the policy. Here it is in action:

```python
    policy = {
        "Version": "2012-10-17", 
        "Statement": [{
            "Sid": "ReadOnlyCloudTrail",
            "Effect": "Deny", 
            "NotAction": "cloudtrail:De*", 
            "Resource": "*"
        }]
    }

    get_iam_info()
    for statement_actions in get_policy_actions(policy):
        statement_actions = sorted(statement_actions, key=lambda x: x["actionName"])
        for action in statement_actions:
            print(f'{action["actionName"]:40} {", ".join(action["actionGroups"])}')
```

This will give the following list:

```
DeleteChannel                            ReadWrite
DeleteEventDataStore                     ReadWrite
DeleteResourcePolicy                     ReadWrite
DeleteServiceLinkedChannel               ReadWrite
DeleteTrail                              ReadWrite
DeregisterOrganizationDelegatedAdmin     ReadWrite
DescribeQuery                            ReadOnly, ReadWrite
DescribeTrails                           ReadOnly, ReadWrite
```

You can see that this policy allows some actions that are not read-only. You could use this code in tests being evaluated in your pipeline to make sure you never accidentally allow non-readonly actions in this policy.

## Future

I will make an installable Python package from the API part. Also, I have some ideas for some neat policy tools:

- check for common mistakes in policies.

- generate readonly service statements for in a permissions boundary