Package: py2sfn-task-tools
py2sfn-task-toolsTools for tasks embedded in anAWS Step Functionsstate machine. This is a helper library forpy2sfn.Features:Offload state data to DynamoDB/S3 instead of storing data in theveryconstrained state machine input data objectCancel the currently executing workflowTable of Contents:InstallationGuideStopping the executionWorking with the State Data ClientDevelopmentInstallationpy2sfn-task-tools requires Python 3.6 or above. It should be installed in apy2sfn task entry point.pipinstallpy2sfn-task-toolsGuideOnce the py2sfn-task-tools library is installed, aContextshould be created and passed to the tasks. Each py2sfn task will then have acontextobject to work with.Stopping the executionIf you need to stop/cancel/abort the execution from within a task, you can use thecontext.stop_executionmethod within your task'srunmethod. A common use case is if you need to check the value of a feature flag at the beginning of the execution and abort if it's false. For example:ifnotsome_condition:returnawaitcontext.stop_execution()You can provide extra detail by passingerrorandcausekeyword arguments to thestop_executionmethod. Theerroris a short string like a code or enum value whereascauseis a longer description.Working with the State Data ClientOne of the stated Step Functions best practices is to avoid passing large payloads between states; the input data limit is only 32K characters. To get around this, you can choose to store data from your task code in a DynamoDB table. With DynamoDB, we have an item limit of 400KB to work with. When you put items into the table you receive a pointer to the DynamoDB item which you can return from your task so it gets includes in the input data object. From there, since the pointer is in thedatadict, you can reload the stored data in a downstream task. This library'sStateDataClientclass provides methods for putting and getting items from this DynamoDB table. It's available in your task'srunmethod ascontext.state_data_client.The client methods are split between "local" and "global" variants. Local methods operate on items stored within the project whereas global methods can operate on items that were stored from any project. Global methods require a fully-specified partition key (primary key, contains the execution ID) and table name to locate the item whereas local methods only need a simple key because the partition key and table name can be infered from the project automatically. Theput_*methods return a dict with metadata about the location of the item, including thekey,partition_key, andtable_name. If you return this metadata object from a task, it will get put on thedataobject and you can call aget_*method later in the state machine.Many methods also accept an optionalindexargument. This argument needs to be provided when getting/putting an item that was originally stored as part of aput_itemsorput_global_itemscall. Providing theindexis usually only done within a map iteration task.Below are a few of the more common methods:put_item/put_itemsTheput_itemmethod puts an item in the state store. It takeskey,data, andindexarguments. For example:context.state_data_client.put_item("characters",{"name":"jerry"})context.state_data_client.put_item("characters",{"name":"elaine"},index=24)Note that the item at the given array index doesn't actually have to exist in the table before you callput_item. However, if it doesn't exist then you may have a fan-out logic bug upstream in your state machine.Theput_itemsmethod puts an entire list of items into the state store. Each item will be stored separately under its corresponding array index. For example:context.state_data_client.put_items("characters",[{"name":"jerry"},{"name":"elaine"}])get_itemTheget_itemmethod gets the data attribute from an item in the state store. It takeskeyandindexarguments. For example:context.state_data_client.get_item("characters")# -> {"name": "jerry"}context.state_data_client.get_item("characters",index=24)# -> {"name": "elaine"}get_item_for_map_iteration/get_global_item_for_map_iterationTheget_item_for_map_iterationmethod gets the data attribute from an item in the state store using theeventobject. This method only works when called within a map iterator task. For example, if theput_itemsexample above was called in a task, and its value was given to a map state to fan out, we can use theget_item_for_map_iterationmethod within our iterator task to fetch each item:# Iteration 0:context.state_data_client.get_item_for_map_iteration(event)# -> {"name": "jerry"}# Iteration 1:context.state_data_client.get_item_for_map_iteration(event)# -> {"name": "elaine"}This works because the map iterator state machine receives an input data object with the schema:{"items_result_table_name":"<DynamoDB table for the project>","items_result_partition_key":"<execution ID>:characters","items_result_key":"characters","context_index":"<array index>","context_value.$":"1"}Theget_item_for_map_iterationis a helper method that uses that input to locate the right item. Theget_global_item_for_map_iterationmethod has the same signature. It should be called when you know that the array used to fan out could have come from another project (e.g. the map state is the first state in a state machine triggered by a subscription).DevelopmentTo run functional tests, you need an AWS IAM account with permissions to:Create/update/delete a DynamoDB tableCreate/update/delete an S3 bucketSet the following environment variables:AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYAWS_DEFAULT_REGIONTo run tests:tox