Serverless number crunching
By: Andrey Shornikov
Tags: AWS, AWS Lambda, Python

General architecture

For high flexibility of our front end we will use quasi static web site. The “ quasi static” means that it does not have any server-side rendering, and only uses minimum client side javascript to query the API and update graphs. This allows to have very simple and fast front end. Regarding hosting — anything will do if it allows javascript. A storage on Amazon S3 with web-hosting option enabled for the bucket will suffice(see static hosting on S3).

Since we took graphics and UI off the backend, the only tasks it has are to ingest the input, crunch the numbers and provide response. AWS lambda function can do all of that, as long as we fit within its limitations. Benefits include high scalability, no need to maintain servers, and Lambda Free tier (which unlike general AWS free tier is available not only to the new users and does not expire after 1 year). The key question is whether your application fits into lambda use case, and since we are talking about actual number crunching, and not just querying pickled ML model — it is not always the case. So, what are the main limitations? Here are the most important:

Limitation on execution time of 900 seconds (beware default value is only 3 seconds, my benchmark code hits that limit rather easily )

Memory limits 128 to 3008 Mb

Dependencies (default runtime has almost no packages preinstalled, managing dependencies is a bit of a hassle, max deployment package 250 Mb unzipped)

Full list of limitations on AWS

By default AWS Lambda Python runtime includes a bare minimum of libraries. Anything that does any simulation using Python will likely require some of the common libraries such as numpy, Scipy or ML libraries like scikit-learn, pyTorch etc.

Getting dependencies work on AWS lambda is not as straightforward as on some other platforms like Azure or GCP, where a simple requirements.txt will do the job.

On AWS you either need to provide locally installed libraries or use available through lambda layers. For the locally installed option you will need to provide libraries in a zipped folder together with the rest of your code. Make sure that the folder size limitation of 256 Mb is respected, and your code and dependencies run smooth with the rest of the lambda runtime environment.

The best way of doing so is to download a docker image of amazon Linux and test everything in there. This topic is well covered by some other authors (a basic instruction , more comprehensive versions with a bit more of Docker)

We will follow simpler approach with lambda layers. Layers are prebuild bundles of dependencies, and if you use ones from AWS itself, they are guaranteed to run smooth with the rest of the runtime. If a single layer does not cover your needs you can use up to five. With our approach of leaving only number crunching in the lambda function the list of our dependencies can be reduced to a workable minimum. More on layers can be found in this introduction post.

In this example I consider a code that tracks time evolution of some Markovian process, where each object in state n has probabilities to either remain as is or change to states n-1 or n+1. We are looking at the time evolution in the population of states in an ensemble of such objects.

From this general description you see that this is a very common task, which may describe anything, from a chemical reaction to a balance between species in an ecosystem.

In my particular case it is a process called charge breeding, and it is found in large research labs working on nuclear physics around the globe such as CERN (CH), Brookhaven National Lab(NY, US) and a few more. It is not important to understand the AWS part, just bragging :)

What is important is that from the description we see that the full extent of required libraries is quite limited and numpy for arrays handling and Scipy for solving differential equations are the only things we need.

This combo is readily available as a single layer on AWS as AWSLambda-Python37-SciPy1x layer for Python 3.7 runtime, for example.

Finally, we will need to bind the lambda function to our front end. Though lambdas can be called by a direct API call, this will require AWS credentials and in standard practice lambdas are called by other AWS events, such as CloudWatch alarms, or in this type of situation by a dedicated AWS service — API Gateway. API Gateway allows you to access your lambda by a simple http request such as GET, POST and get a response.

After all our architecture looks as follows:

Detailed application scheme, drawn by author

Front end is hosted in an S3 bucket, and uses jquery to talk to API endpoint, Bokeh to plot graphs and bootstrap to put our UI in a column based flexible layout. I use optional AWS route 53 to provide DNS alias to the bucket, but for test purposes a meaningful S3 bucket name is as good.

Then we have API gateway to bind together front end and back end. And finally in the back end we have our code that does main simulations, raw data in elements.json and a file with lambda function handler. All loaded as a zipped folder. Dependencies are met through a single lambda layer with numpy and scipy.