Moving your cron jobs to AWS lambda with SAM cli 
Tags: aws, lambdas, cron
Published: 2019-03-31 02:35:59

---
title: Moving your cron jobs to AWS lambda with SAM cli 
published: true
description: 
tags: aws, lambdas, cron
cover_image: https://thepracticaldev.s3.amazonaws.com/i/oymb11c5p6j7xn499od2.png
---

Every now and then, you need to a script to run at regular interval and do something interesting. Before the advent of serverless computing, cron jobs were the defacto way of doing this. But now we have several options that are more cost effective. [ECS scheduled tasks](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduling_tasks.html) and [lambdas](https://aws.amazon.com/lambda/) are two serverless alternatives for task scheduling in AWS. Depending on your workload you can decide which option would be the right choice for you. Lambdas, with 15-minute runtime limit are ideal for small workloads. Let's explore how we can get this working.

## Install SAM cli
AWS Sam cli is a tool with an adorable chipmunk as mascot. It simplifies building and deploying lambdas. 

```bash
pip install --user aws-sam-cli
```

## Create a project

```bash
sam init --name lambda-cron --runtime python3.7
[+] Initializing project structure...

Project generated: ./lambda-cron

Steps you can take next within the project folder
===================================================
[*] Invoke Function: sam local invoke HelloWorldFunction --event event.json
[*] Start API Gateway locally: sam local start-api

Read lambda-cron/README.md for further instructions

[*] Project initialization is now complete
```  

Lets take a look at the generated code

```bash
tree lambda-cron/
lambda-cron/
|-- README.md
|-- event.json
|-- hello_world
|   |-- __init__.py
|   |-- __pycache__
|   |   |-- __init__.cpython-37.pyc
|   |   `-- app.cpython-37.pyc
|   |-- app.py
|   `-- requirements.txt
|-- template.yaml
`-- tests
    `-- unit
        |-- __init__.py
        |-- __pycache__
        |   |-- __init__.cpython-37.pyc
        |   `-- test_handler.cpython-37.pyc
        `-- test_handler.py

5 directories, 12 files
```

This might seem an overkill of artifacts compared to a line in your cron tab. But if you think you are getting rid of the entire server, this doesn't look that unreasonable. Let's take a look at the non-obvious files.


### event.json
sam's scaffolding is targets  the most popular lambda use case - a function servicing an API gateway endpoint. We can ignore this.


### app.py
This file is of primary importance, as this is where your code goes. By default, `lambda_handler` is the name of the function which you populate. For now, we will just limit it to a printing a string.


```python
def lambda_handler(event, context):
    print("Do something interesting!")
```

### requirements.txt
Standard stuff - your package dependencies go here.


### template.yaml
This file is responsible for orchestrating the serverless-ness. It declaratively specifies what components make up your `stack`, where the stack is your entire software architecture deployed as one unit in AWS CloudFromation. If this is not magic, I don't know what is. I am a big fan of CloudFormation.

Note this file was generated to build a lambda function that gets triggered by an HTTP request to the API gateway endpoint. But our goal is to run it at scheduled intervals. Take a look at the section which describes the events.

```yaml
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get
```

We need to replace it with the schedule information. Sam template [supports](https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#schedule) schedule expression in both `cron` syntax and a simpler `rate` expression.

Every minute using rate
```yaml
      Events:
        HelloWorld:
          Type: Schedule
          Properties:
            Schedule: rate(1 minute)
```

Every minute using cron 
```yaml
      Events:
        HelloWorld:
          Type: Schedule
          Properties:
            Schedule: cron(1 * * * ? *)
```
 

### test_handler.py
This is pre-generated unittest stubs for pytest. You can install pytest (`pip install pytest`) and run `pytest` like usual.


## Build
With your tests passing, it is now time to do the builds

```bash
cd lambda-cron/
t lambda-cron $ sam build --use-container
2019-03-30 19:42:24 Starting Build inside a container
2019-03-30 19:42:24 Building resource 'HelloWorldFunction'

Fetching lambci/lambda:build-python3.7 Docker container image......
2019-03-30 19:42:25 Mounting /Users/t/projects/python/playground/lambda-deep-dive/lambda-cron/hello_world as /tmp/samcli/source:ro inside runtime container

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Package: sam package --s3-bucket <yourbucket>

Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
```
Note the first time you run this, it will download the docker image it needs for running.


## Local testing
One of the best functionality that sam offers is the ability to test your code locally in a docker container. This makes quick iteration possible without having to acutally deploy on AWS lambda environment.

```bash
sam local invoke --no-event -t template.yaml
2019-03-30 21:01:43 Found credentials in shared credentials file: ~/.aws/credentials
2019-03-30 21:01:43 Invoking app.lambda_handler (python3.7)

Fetching lambci/lambda:python3.7 Docker container image....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
2019-03-30 21:02:53 Mounting /Users/t/projects/python/playground/lambda-deep-dive/lambda-cron/hello_world as /var/task:ro inside runtime container
START RequestId: 52fdfc07-2182-154f-163f-5f0f9a621d72 Version: $LATEST
Do something intersting!
END RequestId: 52fdfc07-2182-154f-163f-5f0f9a621d72
REPORT RequestId: 52fdfc07-2182-154f-163f-5f0f9a621d72	Duration: 16.52 ms	Billed Duration: 100 ms	Memory Size: 128 MB	Max Memory Used: 22 MB
null
```

## Package your code
Let's create a bucket and pass it to the packaging command. Note we are passing a file called packaged.yaml which is the generated cloud formation template that we will be using in the next command to deploy it.


```bash
aws s3 mb s3://lambda-cron-bucket
sam package --s3-bucket lambda-cron-bucket --output-template-file packaged.yaml

```

## Deploy
Finally deploy your cron job. You'll need to provide a name for your stack and tell cloudformation explicitly that you are ok with the creation of IAM related resources by passing the `CAPABILITY_IAM` flag.

```bash
aws cloudformation deploy --template-file /absolute/path/packaged.yaml --stack-name lambda-cron --capabilities CAPABILITY_IAM
```

## Validate
Lets take a look if our cron is executing every minute.

```bash
sam logs -n HelloWorldFunction --stack-name lambda-cron --tail
```
You should see something like the following:

```bash
2019-03-30 20:35:19 Found credentials in shared credentials file: ~/.aws/credentials
2019/03/31/[$LATEST]7cc1e71b6eb14c89881cb91d6c29b920 2019-03-31T01:29:12.613000 START RequestId: c2c8a898-dfab-450f-944b-d43d77741c35 Version: $LATEST
2019/03/31/[$LATEST]7cc1e71b6eb14c89881cb91d6c29b920 2019-03-31T01:29:12.618000 Do something intersting!
2019/03/31/[$LATEST]7cc1e71b6eb14c89881cb91d6c29b920 2019-03-31T01:29:12.638000 END RequestId: c2c8a898-dfab-450f-944b-d43d77741c35
2019/03/31/[$LATEST]7cc1e71b6eb14c89881cb91d6c29b920 2019-03-31T01:29:12.638000 REPORT RequestId: c2c8a898-dfab-450f-944b-d43d77741c35	Duration: 24.83 ms	Billed Duration: 100 ms 	Memory Size: 128 MB	Max Memory Used: 63 MB
2019/03/31/[$LATEST]7cc1e71b6eb14c89881cb91d6c29b920 2019-03-31T01:30:12.272000 START RequestId: 93645e34-850b-47b0-a4a3-00d10e502057 Version: $LATEST
2019/03/31/[$LATEST]7cc1e71b6eb14c89881cb91d6c29b920 2019-03-31T01:30:12.278000 Do something intersting!
2019/03/31/[$LATEST]7cc1e71b6eb14c89881cb91d6c29b920 2019-03-31T01:30:12.298000 END 
...
```

Congratulations on your first serverless cron job!


## What's with the cover picture?
Keeping with the theme of my other dev.to posts, it has nothing to do with the post. It's view of our beautiful city, Chicago, from my balcony.