Package: async-lambda-unstable
async-lambdaasync-lambdais a framework for creatingAWS Lambdaapplications with built
in support for asynchronous invocation via a SQS Queue. This is useful if you
have workloads you need to split up into separate execution contexts.async-lambdaconverts your application into aServerless Application Model (SAM)template which can be deployed with theSAMcli tool.Anasync-lambdaapplication is comprised of acontrollerobject andtasks.importjsonfromasync_lambdaimportAsyncLambdaController,ScheduledEvent,ManagedSQSEvent,config_set_nameapp=AsyncLambdaController()config_set_name("project-name")lambda_handler=app.async_lambda_handler# This "export" is required for lambda.@app.scheduled_task('ScheduledTask1',schedule_expression="rate(15 minutes)")defscheduled_task_1(event:ScheduledEvent):app.async_invoke("AsyncTask1",payload={"foo":"bar"})# Payload must be JSON serializable and < 2560Kb@app.async_task('AsyncTask1')defasync_task_2(event:ManagedSQSEvent):print(event.payload)#{"foo": "bar"}When the app is packaged for lambda, only the main module, and thevendorandsrcdirectories are included.TasksThe core abstraction inasync-lambdais atask. Each task will result in a separate lambda function.
Tasks have atrigger_typewhich determines what event triggers them. A task is identified by its uniquetask_id.All task decorators share common arguments for configuring the underlying lambda function:memory: int = 128Sets the memory allocation for the function.timeout: int = 60Sets the timeout for the function (max 900 seconds).ephemeral_storage: int = 512Sets the ephemeral storage allocation for the function.maximum_concurrency: Optional[int] = NoneSets the maximum concurrency value for the SQS trigger for the function. (only applies toasync_taskandsqs_tasktasks.)Async TaskAll async tasks have a matching SQS queue which the lambda function consumes from (1 message per invocation).
All async task queues share a DLQ. Async tasks can be invoked from anywhere within the app by using theAsyncLambdaController.async_invokemethod. Calling this method sends a message into the queue for the given task.
The task function should have a single parameter of theManagedSQSEventtype.app=AsyncLambdaController()@app.async_task("TaskID")defasync_task(event:ManagedSQSEvent):event.payload# payload sent via the `async_invoke` methodevent.source_task_id# the task_id where the event originatedIt is quite easy to get into infinite looping situations when utilizingasync-lambdaand care should be taken.INFINITE LOOP EXAMPLE# If task_1 where to ever get invoked, then it would start an infinite loop with# task 1 invoking task 2, task 2 invoking task 1, and repeat...@app.async_task("Task1")deftask_1(event:ManagedSQSEvent):app.async_invoke("Task2",{})@app.async_task("Task2")deftask_1(event:ManagedSQSEvent):app.async_invoke("Task1",{})Unmanaged SQS TaskUnmanaged SQS tasks consume from any arbitrary SQS queue (1 message per invocation).
The task function should have a single parameter of theUnmanagedSQSEventtype.app=AsyncLambdaController()@app.sqs_task("TaskID",queue_arn='queue-arn')defsqs_task(event:UnmanagedSQSEvent):event.body# sqs event bodyScheduled TaskScheduled tasks are triggered by an eventbridge schedule. The schedule expression can be
acron expressionor arate expression.
The task function should have a single parameter of theScheduledEventtype.app=AsyncLambdaController()@app.scheduled_task("TaskID",schedule_expression='rate(15 minutes)')defscheduled_task(event:ScheduledEvent):...API TaskAPI tasks are triggered by a Web Request.async-lambdacreates an APIGateway endpoint matching themethodandpathin the task definition. It is possible to configure a custom domain and certificate
for all API tasks within anasync-lambdaapp.
The task function should have a single parameter of theAPIEventtype.app=AsyncLambdaController()@app.api_task("TaskID",path='/test',method='get')defapi_task(event:APIEvent):event.headers# request headersevent.querystring_params# request querystring paramsevent.body# request bodyasync-lambdaconfigConfiguration options can be set with the.async_lambda/config.jsonfile.
The configuration options can be set at the app, stage, and task level. A configuration option set
will apply unless overridden at a more specific level (app -> stage -> task -> stage).
The override logic attempts to non-destructive so if you have alayersof['layer_1']at the app level,
and[layer_2]at the stage level, then the value will be['layer_1', 'layer_2'].Config file levels schema{
    # APP LEVEL
    "stages": {
        "stage_name": {
            # STAGE LEVEL
        }
    },
    "tasks": {
        "task_id": {
            # TASK LEVEL
            "stages": {
                "stage_name": {
                    # TASK STAGE LEVEL
                }
            }
        }
    }
}At any of theselevelsany of the configuration options can be set:With the exception ofdomain_name,tls_version, andcertificate_arnwhich can not be set at the task level.environment_variables{
    "ENV_VAR_NAME": "ENV_VAR_VALUE"
}This config value will set environment variables for the function execution.
These environment variables will also be available during build time.The value is passed to theEnvironmentproperty onSAM::Serverless::Functionpolicies[
    'IAM_POLICY_ARN' | STATEMENT
]Use this config option to attach any arbitrary policies to the lambda functions execution role.The value is passed to thePoliciesproperty onSAM::Serverless::Function, in addition to theasync-lambdacreated inline policies.layers[
    "LAYER_ARN"
]Use this config option to add any arbitrary lambda layers to the lambda functions. Ordering matters,
and merging is done thru concatenation.The value is passed to theLayersproperty onSAM::Serverless::Functionsubnet_ids[
    "SUBNET_ID
]Use this config option to put the lambda function into a vpc/subnet.The value is passed into theSubnetIdsfield of theVpcConfigproperty onSAM::Serverless::Functionsecurity_group_ids[
    "SECURITY_GROUP_ID"
]Use this config option to attach a security group to the lambda function.The value is passed into theSecurityGroupIdsfield of theVpcConfigproperty onSAM::Serverless::Functionmanaged_queue_extras[
    {
        # Cloudformation resource
    }
]Use this config option to add extra resources for managed SQS queues (async_tasktasks.)For example this might be used to attach alarms to these queues.Each item in the list should be a complete cloudformation resource.async-lambdaprovides a few custom substitutions
so that you can reference the extras and the associated managed sqs resource byLogicalId.$QUEUEID"will be replaced with theLogicalIdof the associated Managed SQS queue.$EXTRA<index>will be replaced with theLogicalIdof the extra at the specified index.domain_nameThis config value can only be set at the app or stage level."domain_name"If yourasync-lambdaapp contains anyapi_tasktasks, then aAWS::Serverless::Apiresource is created.This config value will set theDomainNamefield of theDomainpropertytls_versionThis config value can only be set at the app or stage level."tls_version"If yourasync-lambdaapp contains anyapi_tasktasks, then aAWS::Serverless::Apiresource is created.This config value will set theSecurityPolicyfield of theDomainpropertyPossible values areTLS_1_0andTLS_1_2certificate_arnThis config value can only be set at the app or stage level."certificate_arn"If yourasync-lambdaapp contains anyapi_tasktasks, then aAWS::Serverless::Apiresource is created.This config value will set theCertificateArnfield of theDomainpropertyBuilding anasync-lambdaappWhen the app is packaged for lambda, only the main module, and thevendorandsrcdirectories are included.From the project root directory, utilize theasync-lambdaCLI tool to generate
a SAM template and function bundle. Optionally specify thestageto usestagespecific configs.# app.py contains the root AsyncLambdaControllerasync-lambdabuildapp--stage<stage-name>This will generate a SAM templatetemplate.jsonas well as andeployment.zipfile.
This template and zip file can then be deployed or transformed into regular cloudformation
via thesamorawscli tools.Known LimitationsLambda Configuration - not all of the lambda configuration spec is present inasync-lambda. It is relatively trivial to add in configuration options. Make an issue if there is a feature you would like to see implemented.Theasync_invokepayload must beJSONserializable withjson.dumps.It is possible to get into infinite loops quite easily. (Task A invokes Task B, Task B invokes Task A)