QuickStart: spaCy on AWS Lambda Up and Running in 10 Minutes | by Sean Zhai | Towards Data Science

Let’s keep our promise and get it done first. Further discussion follows the QuickStart section if you want to read further. I hope this arrangement could save you time.
Log in to the AWS account. Create a new Lambda function. Choose Python 3.8, and use default settings.
Go to Lambda [Configuration] [Designer], and click on Layers. Then click on [Add a layer] on the lower panel.
Choose [Specify an ARN]. Use the following ARN for spaCy, then click [Add]
arn:aws:lambda:us-west-1:770693421928:layer:Klayers-python38-spacy:30
Repeat the above step and add Language Model as Layer, using the following ARN.
arn:aws:lambda:us-west-1:770693421928:layer:Klayers-python38-spacy_model_en_small:1
The Layers should look like this when finished.
Allocate more memory for spaCy in Lambda. Choose 512 MB for Memory, and 10 sec for Timeout.
Import spacy into Lambda. Let’s write a Hello World function to test our layers are working properly.
import jsonimport spacydef lambda_handler(event, context):    nlp = spacy.load("/opt/en_core_web_sm-2.2.5/en_core_web_sm/en_core_web_sm-2.2.5")    doc = nlp("Hello World from spaCy")    return {        'statusCode': 200,        'body': json.dumps(doc.text)    }
The result should look like this. As you might notice, the default 128 MB memory allocation is not enough, as a Hello World program used 200 MB.
Congratulation! Now spaCy is ready to use. Feel free to get back to your busy days if you’ve got what you need.
If you have time, I would like to share my experience with NLP (Natural Language Processing).
Actually, I first started NLP (Natural Language Processing) with TextBlob, which feels like something can be learned during a sunny afternoon. TextBlob has a handy noun phrase extractor, and it does sentiment analysis with ease. I used TextBlob for a data visualization project and it was fun to get some quick results.
Since April, I started working on a computational poetry project, and it required an in-depth analysis of complex literature. I found TextBlob not sophisticated enough to handle long and complicated structures. It was then I switched to spaCy.
spaCy is designed to help you do real work — to build real products, or gather real insights. The library respects your time, and tries to avoid wasting it. It’s easy to install, and its API is simple and productive. We like to think of spaCy as the Ruby on Rails of Natural Language Processing. — spacy.io
I’m not sure about whether the Ruby on Rails analogy still makes sense, but spaCy has worked quite well for me. Take a look at spaCy Universe, it is quite impressive.
My goal is to develop a cloud-based application, so I need to run spaCy on the cloud. Lambda is the most cost-effective choice on AWS, as it is billed only for the time a request comes in. A virtual machine on AWS will be charged all its uptime. Being a powerful package, spaCy is quite large, its default distribution exceeds the 250 MB limit required by Lambda.
People have taken extra efforts to reduce the size of spaCy, and there are DIY instructions to fit spaCy into Lambda, but it takes some serious effort if you are determined to carry it through.
Luckily, Keith Rozario made an ARN ready to use as AWS Layers. This is what you’ve seen in the QuickStart section in this writing. If you really timed these steps with a stopwatch, your result might be well below 10 minutes.