AWS Step Functions, Intrinsic Functions In-Depth | Serverless
Tags: serverless, aws, devops, productivity
Published: 2023-03-07 17:39:03

AWS Step Functions enables you to orchestrate serverless workflows by integrating with different AWS services.

> 📋 **Note:** If you want to read more about AWS Step Functions before starting this article you can read [AWS Step Functions In-Depth | Serverless](https://dev.to/aws-builders/aws-step-functions-in-depth-serverless-5e4i)

The main parts of this article:
1. About Intrinsic functions
2. Examples

### 1. what are Intrinsic functions?
Intrinsic functions allows you to run directly on your state machine some simple processing.

Well, let's take a look at how we used to manipulate our data in Step Functions. In order to do so, we needed a state and a Lambda function which would take the data as input, update it, and make all the necessary changes before passing the data to the next state. However, upon closer examination, it became clear that our architecture required excessive resources and involved too many steps.

The Amazon States Language provides several intrinsic functions, also known as intrinsics, that can help you perform basic data processing operations without using a Task state. These intrinsics are constructs that resemble functions in programming languages and can assist payload builders in processing the data that goes to and from the Resource field of a Task state.

Using intrinsic functions can help you build applications where you don't have to rely on Lambda functions, which can save you time and money, reduce complexity (by consolidating logic in one place), minimize throughput, and reduce the payload between different tasks.

### Examples
Now let's build simple examples and try all the options that we have on Intrinsic functions, the following are the different operations that we will cover

- #### Intrinsics for arrays: `States.Array`, `States.ArrayPartition`, `States.ArrayContains`, `States.ArrayRange`, `States.ArrayGetItem`, `States.ArrayLength`, `States.ArrayUnique`

- #### Intrinsics for data encoding and decoding: `States.Base64Encode`, `States.Base64Decode`

- #### Intrinsic for hash calculation: `States.Hash`
- #### Intrinsics for JSON data manipulation: `States.JsonMerge`, `States.StringToJson`, `States.JsonToString`
- #### Intrinsics for Math operations: `States.MathRandom`, `States.MathAdd`
- #### Intrinsic for String operation: `States.StringSplit`
- #### Intrinsic for unique identifier generation: `States.UUID`
- #### Intrinsic for generic operation: `States.Format`

Before adding intrinsic functions, let's first build our Step Functions. To keep it simple, we will have only two states, and a successful return at the end. Furthermore, our Lambda functions for each state will be simple, only logging out the data and passing the value to the next state.

```yml
firstLambdaARN: &FIRST_ARN arn:aws:lambda:${env:region}:${env:accountId}:function:${self:service}-${env:stage}-firstState
secondLambdaARN: &SECOND_ARN arn:aws:lambda:${env:region}:${env:accountId}:function:${self:service}-${env:stage}-secondState
states:
  IntrinsicExample:
    name: IntrinsicExample
    definition:
      Comment: "Intrinsic Functions Example"
      StartAt: firstState
      States:
        firstState:
          Type: Task
          Resource: *FIRST_ARN
          Next: secondState
        secondState:
          Type: Task
          Resource: *SECOND_ARN
          Next: success
        success:
          Type: Succeed
```

```javascript
"use strict";

module.exports.firstState = async (event) => {
  console.log('firstState event =>', event);

  return event;
};

module.exports.secondState = async (event) => {
  console.log('secondState event =>', event);

  return {
    status: 'SUCCESS'
  };
};
```

Everything seems ready, let's start adding our intrinsic functions one by one and start seeing the results.

> 📋 **Note:** I'm not going with description of each intrinsic function, if you want more documentation how each one is working you can visit the link [Intrinsic functions](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html)

**To use intrinsic functions you must specify .$ in the key value in your state machine definitions.**

**You can nest up to 10 intrinsic functions within a field in your workflows.**

____

#### Intrinsics for arrays

In the first one we are going to pass the whole payload as it is to `input` also we are going to get the length. Inside the state we can see that `Parameters` are now added 
- `States.ArrayLength`
```yml
firstLambdaARN: &FIRST_ARN arn:aws:lambda:${env:region}:${env:accountId}:function:${self:service}-${env:stage}-firstState
secondLambdaARN: &SECOND_ARN arn:aws:lambda:${env:region}:${env:accountId}:function:${self:service}-${env:stage}-secondState
states:
  IntrinsicExample:
    name: IntrinsicExample
    definition:
      Comment: "Intrinsic Functions Example"
      StartAt: firstState
      States:
        firstState:
          Type: Task
          Resource: *FIRST_ARN
          Parameters:
            input.$: "$"
            length.$: "States.ArrayLength($.inputArray)"
          Next: secondState
        secondState:
          Type: Task
          Resource: *SECOND_ARN
          Next: success
        success:
          Type: Succeed
```

Perfect! Now we need to execute our Step Function. The easiest way to do so is by creating a simple function that can be triggered from an API call. We can then use the AWS SDK to run our Step Function.
```javascript
"use strict";
const { StepFunctions } = require('aws-sdk');
const stepFunctions = new StepFunctions();

module.exports.runIntrinsicFunction = async (event) => {
  try {
    const stepFunctionResult = stepFunctions.startExecution({
      stateMachineArn: process.env.INTRINSIC_EXAMPLE_STEP_FUNCTION_ARN,
      input: JSON.stringify({
        inputArray: [
          {
            id: 1,
            title: 'book',
          },
          {
            id: 2,
            title: 'pen',
          },
          {
            id: 3,
            title: 'watch',
          },
          {
            id: 4,
            title: 'laptop',
          }
        ]
      })
    }).promise();
    console.log('stepFunctionResult =>', stepFunctionResult);

    return {
      statusCode: 200,
      body: JSON.stringify({
        message: `This is test API`,
      }, null, 2),
    };
  } catch (error) {
    console.log(error);
  }
};
```
Now we can see that our first state takes the following input. It's amazing to see how powerful this is and how it can help minimize our code logic. With this new functionality, we can simplify our code and make it more efficient. Let's now move on to the other examples.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v5wbidwfhm2kndzz1nfd.png)

- `States.Array`, `States.ArrayPartition`

Payload:
```javascript
input: JSON.stringify({
  inputInteger: 12345678,
  inputForPartition: [1, 2, 3, 4, 5, 6, 7, 8]
})
```

Intrinsic:
```yml
Parameters:
  buildId.$: "States.Array($.inputInteger)"
  partition.$: "States.ArrayPartition($.inputForPartition, 2)"
```

Result:
```json
{
  "partition": [
    [
      1,
      2
    ],
    [
      3,
      4
    ],
    [
      5,
      6
    ],
    [
      7,
      8
    ]
  ],
  "BuildId": [
    12345678
  ],
}
```

- `States.ArrayContains`, `States.ArrayRange`

Payload:
```javascript
input: JSON.stringify({
  inputArray: [1,2,3,4,5,6,7,8,9],
  lookingFor: 8
})
```

Intrinsic:
```yml
Parameters:
  containsResult.$: "States.ArrayContains($.inputArray, $.lookingFor)"
  rangeResult.$: "States.ArrayRange(1, 20, 3)"
```

Result:
```json
{
  "containsResult": true,
  "rangeResult": [
    1,
    4,
    7,
    10,
    13,
    16,
    19
  ]
}
```

- `States.ArrayGetItem`, `States.ArrayUnique`

Payload:
```javascript
input: JSON.stringify({
  inputArray: [1,2,3,3,3,3,3,3,4],
  index: 5,
})
```

Intrinsic:
```yml
Parameters:
  getItemResult.$: "States.ArrayGetItem($.inputArray, $.index)"
  uniqueResult.$: "States.ArrayUnique($.inputArray)"
```

Result:
```json
{
  "getItemResult": 3,
  "uniqueResult": [
    1,
    2,
    3,
    4
  ]
}
```

____

#### Intrinsics for data encoding and decoding
- `States.Base64Encode`, `States.Base64Decode`

Payload
```javascript
input: JSON.stringify({
  input: "Data to encode",
  base64: "RGF0YSB0byBlbmNvZGU="
})
```

Intrinsic:
```yml
Parameters:
  encodeResult.$: "States.Base64Encode($.input)"
  decodeResult.$: "States.Base64Decode($.base64)"
```

Result:
```json
{
  "decodeResult": "Data to encode",
  "encodeResult": "RGF0YSB0byBlbmNvZGU="
}
```

____

#### Intrinsic for hash calculation
- `States.Hash`
The hashing algorithm: MD5, SHA-1, SHA-256, SHA-384, SHA-512

Payload:
```javascript
input: JSON.stringify({
  Data: "input data", 
  Algorithm: "SHA-1"
})
```

Intrinsic:
```yml
Parameters:
  result.$: "States.Hash($.Data, $.Algorithm)"
```

Result:
```json
{
  "result": "aaff4a450a104cd177d28d18d74485e8cae074b7"
}
```

____

#### Intrinsics for JSON data manipulation
- `States.JsonMerge`, `States.StringToJson`, `States.JsonToString`

Payload:
```javascript
input: JSON.stringify({
  json1: { "a": {"a1": 1, "a2": 2}, "b": 2, },
  json2: { "a": {"a3": 1, "a4": 2}, "c": 3 },
  escapedJsonString: "{\"foo\": \"bar\"}",
  unescapedJson: {
    foo: "bar"
  }
})
```

Intrinsic:
```yml
Parameters:
  output.$: "States.JsonMerge($.json1, $.json2, false)"
  toJsonResult: "States.StringToJson($.escapedJsonString)"
  toStringResult: "States.JsonToString($.unescapedJson)"
```

Result:
```json
{
  "output": {
    "a": {
      "a3": 1,
      "a4": 2
    },
    "b": 2,
    "c": 3
  },
  "toJsonResult": {
    "foo": "bar"
  },
  "toStringResult": "{\"foo\":\"bar\"}"
}
```

____

#### Intrinsics for Math operations
- `States.MathRandom`, `States.MathAdd`

Payload:
```javscript
input: JSON.stringify({
  start: 1,
  end: 999,
  value: 100,
  step: 2
})
```

Intrinsic:
```yml
Parameters:
  randomResult.$: "States.MathRandom($.start, $.end)"
  additionResult.$: "States.MathAdd($.value, $.step)"
```

Result:
```JSON
{
  "randomResult": 110,
  "additionResult": 102
}
```

____

#### Intrinsic for String operation
- `States.StringSplit`

Payload:
```javascript
input: JSON.stringify({
  inputStringOne: "1,2,3,4,5",
  splitterOne: ",",
  inputStringTwo: "This.is+a,test=string",
  splitterTwo: ".+,="
})
```

Intrinsic:
```yml
Parameters:
  resultOne.$: "States.StringSplit($.inputStringOne, $.splitterOne)"
  resultTwo.$: "States.StringSplit($.inputStringTwo, $.splitterTwo)"
```

Result:
```json
{
  "resultOne": [
    "1",
    "2",
    "3",
    "4",
    "5"
  ],
  "resultTwo": [
    "This",
    "is",
    "a",
    "test",
    "string"
  ]
}
```

____

#### Intrinsic for generic operation
- `States.UUID`

Intrinsic:
```yml
Parameters:
  uuid.$: "States.UUID()"
```

Result:
```json
{
  "uuid": "c23c5e29-e9df-4507-8152-c677792511a4"
}
```

____

#### Intrinsic for generic operation
- `States.Format`

Payload:
```javascript
input: JSON.stringify({
  template: "Hello, this is {}."
})
```

Intrinsic:
```yml
Parameters:
  formatResult.$: "States.Format($.template, $.name)"
```

Result:
```json
{
  "formatResult": "Hello, this is Awedis."
}
```

____

Now lets see how we can nest some intrinsic functions together.

First I want to merge two objects together and then escape it.

Payload:
```javascript
input: JSON.stringify({
  json1: { id: '001', title: 'book' },
  json2: { id: '001', category: 'products', label: 'business' }
})
```

Intrinsic:
```yml
Parameters:
  time.$: "$$.State.EnteredTime"
  output.$: "States.JsonToString(States.JsonMerge($.json1, $.json2, false))"
```

Result:
```json
{
  "output": "{\"id\":\"001\",\"label\":\"business\",\"title\":\"book\",\"category\":\"products\"}",
  "time": "2023-03-05T13:17:18.682Z"
}
```

### Conclusion
Using intrinsic functions can make your work easier since software engineers use these functions almost daily. Intrinsic functions can help minimize and simplify your code, making it much cleaner.

This article is part of the "Messaging with Serverless" series that I have been writing for a while. If you are interested in reading about other serverless offerings from AWS, feel free to visit the links below.