Handle optional input parameters in AWS Step Functions without Lambda
Tags: aws, serverless, javascript, stepfunctions
Published: 2023-06-28 14:15:57

**TL;DR**

If you want to apply data processing to an input object in your state machine without the well known struggle of dealing with optional parameters, you can use the [`HandleOptionalParameters`](https://github.com/pmilliotte/handle-optional-parameters) CDK construct as follows:

```ts
const transformInputValuesTo1ItemArraysTask = new HandleOptionalParameters(
  scope,
  "Handle optional parameters",
  {
      requiredProperties: ["requiredProperty"],
      optionalProperties: ["optionalProperty1", "optionalProperty2"],
      // Apply States.Array intrinsinc function on every input values:
      dataProcessing: "States.Array($.value)",
      // Other examples:

      // Encode every input values and return the object with same input keys but with encoded values:
      // dataProcessing: "States.Base64Encode($.value)",

      // Concatenate the value with the property name, with the JsonPath $.propertyName:
      // dataProcessing: "States.Format('Current key is {} and associated value is {}.', $.propertyName, $.value)",
  }
);

new StateMachine(
  scope,
  "State machine using HandleOptionalParameters construct",
  {
      definition: processObjectValuesTask,
  }
);
```

## 🔴 Context: AWS Step Functions throws if the provided JsonPath is not present

AWS Step Functions lets you do data processing on a state machine execution input thanks to its [intrinsic functions](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html).

For example, let's say you need to put every values of an object input in an array: ![Transform input](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/akcejfx44qjo4wuvirja.png)



To transform a value to an 1-item array containing this value with Amazon State Language, you can use the [`States.Array()`](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html#asl-intrsc-func-arrays) intrinsic function:

![Transform one item object](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/teztq3zoq4vtkp9u0zyp.png)

To transform every values of an input object, you can apply `States.Array()` on every properties:

![Transform multi items object](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lj2a0vfzu3beczqbcebj.png)

However, when dealing with object inputs, you have to make sure that each property you want to process is present in the input, otherwise the execution will fail ❌:

> An error occurred while executing the state 'Transform input values to array' (entered at the event id #2). The function 'States.Array($.optionalProperty)' had the following error: The JsonPath argument for the field '$.optionalProperty' could not be found in the input `{ "requiredProperty": 1 }`

Indeed, if `foo` is actually undefined:

![Failure if some properties are optional](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kw5vkecas3hqh12lakz8.png)

You thus need to handle these optional input parameters.

## 🟠 Choices: the false good idea for large inputs

[Choice](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-choice-state.html) is another common flow state of AWS Step Functions that enables you to decide on which branch to continue the state machine execution, based on a condition evaluated at run time. You can thus use a Choice state to evaluate the optional property's presence in the input, and apply the `States.Array()` function only if the property is defined in the input:

![Avoid failure with choices](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9a8xe7z9j7e4turvblfn.png)

However, if the input has many optional values, your step function may look like this 😓:

![Choices solution is not scalable](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p5akbfktver4dp0tndzz.png)

This causes scalability issues:

- 🧑‍💻 **DX:** as you can see in the execution graph view, debugging is painful in the AWS console
- 🛑 **AWS limitations:** in my case, Cloudformation template hard size limit of 1MB is reached with 10 optional properties
- 💸 **Cost:** in a standard workflow, billing depends on the number of state transition which is proportional to the number of optional properties
- 🚀 **Performance:** Choices are not parallelised which leads speed to decrease with the number of optional properties

You thus need to handle the scalability of your optional parameters.

## 🟢 Let the `HandleOptionalParameters` CDK construct handle the optional parameters for you

The [`HandleOptionalParameters`](https://github.com/pmilliotte/handle-optional-parameters) CDK construct processes every defined values with whatever data processing you provide, while ignoring the undefined ones. And it does it in 9 common flow tasks, regardless of the number of optional parameters 🎉:

![Handle optional parameters the scalable way](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f0opv8t6eq3tuhocs8ti.png)

Let's break it down:


![CDK construct explained](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ifl1rpc99wpj4catzuu4.png)

## Conclusion

From now on, stop using Lambda for handling large state machine inputs with many optional parameters ! 😉