Step Functions with Localstack
Tags: localstack, stepfunctions, aws
Published: 2020-03-30 21:27:21

AWS State Machines are a great tool to co-ordinate the components of a distributed applications and microservices using visual workflows. Using the aws console to build and deploy step functions is great and easy. But it makes local development somewhat slow. So, as usual, I invested some time in trying to make this whole setup work with localstack.

## Table of Contents
* [Tools](#tools)
* [Functions](#functions)
* [State Machine](#state-machine)
* [Run everything](#run-everything)

### Tools <a name="tools"></a>
For this you will need to have,
* [Docker](docker.com)
* Node12.X
* [awscli-local](https://github.com/localstack/awscli-local)

### Functions <a name="functions"></a>
![We are not talking calculus](https://media.giphy.com/media/42HJ26S7bvbmVhcaR2/giphy.gif) <figcaption>We are not talking calculus</figcaption>
By functions, we are talking lambda functions. For our case, we will have 2 functions: `createMatches` and `calculateRunRate`. Both are pretty simple functions. `createMatches` takes in the following input
```Typescript
{
  teamA: string; // Team playing the match
  teamB: string; // 2nd team playing the match
  numMatches: number; // How many matches they are playing?
  matchType: MatchType; // Is it a T20 or ODI?
}
```
and it outputs an array of match results in the form,
```Typescript
{
  matchType: MatchType; // Is it a T20 or ODI?
  teamA: MatchScore; // How many runs scored, wickets lost, overs played?
  teamB: MatchScore;
}
```
`calculateRunRate` then takes in the output of `createMatches` function as input and calculates the run rate for each team and returns it as,
```Typescript
{
  teamA: number; // signed floating number
  teamB: number; // signed floating number
}
```
![Run rates can be heart breaking](https://qph.fs.quoracdn.net/main-qimg-9164bd0866d4e91e1ca4e7bc17bf31d9.webp)<figcaption>Run rates can be heart breaking</figcaption>

### State Machine <a name="state-machine">
Now that we have our lambda functions in place, lets create the state machine. This is what our machine will look like,
```JSON
{
  "Comment": "State Machine to calculate net run rates for given teams.",
  "StartAt": "CreateMatches",
  "States": {
    "CreateMatches": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:CreateMatchesFunction",
      "Next": "CalculateRunRate"
    },
    "CalculateRunRate": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:CalculateRunRateFunction",
      "End": true
    }
  }
}

```
This is a pretty straight-forward json file. The specs for each property can be read [here](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-state-machine-structure.html). The state machine needs to specify a starting point, in this case it is the `CreateMatches` step. And then it also needs to be specify the end step.
![The bowling machine](https://media.giphy.com/media/6XeycPXa9f8s/giphy.gif)<figcaption>The bowling machine</figcaption>

### docker-compose
Now we will write the docker-compose.yml. This is pretty straight-forward, 
```yml
version: '3.7'

services:
  localstack:
    image: localstack/localstack
    environment:
      - DATA_DIR=/tmp/localstack/data
      - SERVICES=lambda
      - TZ=America/New_York
      - LAMBDA_EXECUTOR=docker
    ports:
      - '4574:4574'
    volumes:
      - /tmp/localstack:/tmp/localstack
      - /var/run/docker.sock:/var/run/docker.sock

  stepfunctions:
    container_name: stepfunctions
    image: amazon/aws-stepfunctions-local:latest
    environment:
      - LAMBDA_ENDPOINT=http://localstack:4574
    ports:
      - '8083:8083'
```
Things to note here that we are starting `lambda` service on localstack. More information on localstack can be found [here](https://github.com/localstack/localstack). The service will be available on port `4574`. Also pay attention to the `LAMBDA_EXECUTOR` and shared volumes. We need to do that coz with localstack the lambda executor needs docker to run. Next we have the `stepfunctions` service, which is built using the `amazon/aws-stepfunctions-local:latest` docker image. Here we need to specify the lambda endpoint. This is endpoint where the step functions will look for lambda functions to execute.
![Where is the ball?](https://media.giphy.com/media/meNaovDMpxyUw/giphy.gif)<figcaption>Where is the ball?</figcaption>

### Run everything <a name="run-everything"></a>
In order to run this, we will have to run a few commands which can be put together is a script. To deploy the lambda functions to localstack, we will first need to zip them. You can do that by any zipping tool. I did it via `zip` command line utility.
```sh
zip -r ./createMatches.zip ./createMatchesHandler.js
zip -r ./calculateRunRate.zip ./calculateRunRateHandler.js
```
Next start the containers by running `docker-compose up -d`. Once the containers are up, you can deploy the lambda functions to localstack like this:
```sh
awslocal lambda create-function \
  --function-name CreateMatchesFunction \
  --runtime nodejs12.x \
  --handler createMatchesHandler.handler \
  --role arn:aws:iam::012345678901:role/DummyRole \
  --zip-file fileb://dist/createMatches.zip

awslocal lambda create-function \
  --function-name CalculateRunRateFunction \
  --runtime nodejs12.x \
  --handler calculateRunRateHandler.handler \
  --role arn:aws:iam::012345678901:role/DummyRole \
  --zip-file fileb://dist/calculateRunRate.zip
```
The output of these commands will look something like this,
```
{
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "CodeSha256": "iA5Rqi3x8PvmLWPZBvWXYrpOQC9tx6zi6TjpU9K8uKI=",
    "FunctionName": "CreateMatchesFunction",
    "LastModified": "2020-03-30T21:17:30.828+0000",
    "RevisionId": "70a91d5c-c42d-422e-9e93-fbc8d6ef638c",
    "CodeSize": 995,
    "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:CreateMatchesFunction",
    "Version": "$LATEST",
    "Role": "arn:aws:iam::012345678901:role/DummyRole",
    "Timeout": 3,
    "Handler": "createMatchesHandler.handler",
    "Runtime": "nodejs12.x",
    "Description": ""
}
{
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "CodeSha256": "GF2chlh/6N9mcPAyg6ysY537GsN+fff6zAuNeQUaz3Y=",
    "FunctionName": "CalculateRunRateFunction",
    "LastModified": "2020-03-30T21:17:31.469+0000",
    "RevisionId": "e1bfe0ab-bce5-4e21-b7c4-8a2bfa980a22",
    "CodeSize": 815,
    "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:CalculateRunRateFunction",
    "Version": "$LATEST",
    "Role": "arn:aws:iam::012345678901:role/DummyRole",
    "Timeout": 3,
    "Handler": "calculateRunRateHandler.handler",
    "Runtime": "nodejs12.x",
    "Description": ""
}
```
Ok, so now we have successfully deployed our lambda functions to localstack. Now we just need to create our state machine and execute the functions. For that, we can run the following commands,
```sh
aws stepfunctions \
  --endpoint http://localhost:8083 \
  create-state-machine \
  --definition file://state-machine.json \
  --name "NetRunRateCalculatorStateMachine" \
  --role-arn "arn:aws:iam::012345678901:role/DummyRole"

aws stepfunctions \
  --endpoint http://localhost:8083 \
  start-execution \
  --state-machine arn:aws:states:us-east-1:123456789012:stateMachine:NetRunRateCalculatorStateMachine \
  --name test
  --input "{\"teamA\":\"India\", \"teamB\": \"Australia\", \"numMatches\": 3, \"matchType\": \"t20\"}"
```
This will give you the following output,
```
{
    "creationDate": 1585603052.224,
    "stateMachineArn": "arn:aws:states:us-east-1:123456789012:stateMachine:NetRunRateCalculatorStateMachine"
}
{
    "startDate": 1585603052.766,
    "executionArn": "arn:aws:states:us-east-1:123456789012:execution:NetRunRateCalculatorStateMachine:test"
}
```
Now to check the status of the execution you can run the `describe-execution` utility on step functions, like this,
```sh
aws stepfunctions \
  --endpoint http://localhost:8083 \
  describe-execution \
  --execution-arn arn:aws:states:us-east-1:123456789012:execution:NetRunRateCalculatorStateMachine:test
```
If the machine already completed execution you will get the following output,
```JSON
{
  "status": "SUCCEEDED",
  "startDate": 1585603052.766,
  "name": "test",
  "executionArn": "arn:aws:states:us-east-1:123456789012:execution:NetRunRateCalculatorStateMachine:test",
  "stateMachineArn": "arn:aws:states:us-east-1:123456789012:stateMachine:NetRunRateCalculatorStateMachine",
  "stopDate": 1585603055.931,
  "output": "{\"teamA\":-3.4393976210462283,\"teamB\":3.4393976210462283}
",
  "input": "{\"teamA\":\"India\", \"teamB\": \"Australia\", \"numMatches\": 3, \"matchType\": \"t20\"}"
}
```
That's it. You have successfully created a state machine and run it against the lambdas in localstack. You can find the full code [here](https://github.com/vikasgarghb/net-run-rate-calculator).
![Success...](https://media.giphy.com/media/zsQ4dmaXqQaQw/giphy.gif)<figcaption>Success...</figcaption>