Load a Large spaCy model on AWS Lambda | by Hironsan | Towards Data Science

spaCy is a useful tool that allows us to perform many natural language processing tasks. When integrating spaCy into an existing application, it is convenient to provide it as an API using AWS Lambda and API Gateway. However, due to Lambda’s limitations, it is hard to deploy large models.
In this article, I will show you how to deploy spaCy using the recently released feature to mount Elastic File System (EFS) on AWS Lambda. By using this feature, we can store a large size model to EFS and load it from Lambda functions. Specifically, it is possible to load data larger than the space available in Lambda’s/tmp (512MB).
The overall architecture is as follows. Lambda loads the spaCy package on Lambda Layers. EFS stores the spaCy models. Lambda then loads the models from EFS. For redundancy, four subnets are placed on two different availability zones.
Requirements:
Docker
AWS CLI
First of all, we must configure VPC that can reach the EFS mount targets. Here, we create a VPC, an internet gateway, two NAT gateway, and four subnets(two public, two private).
In the VPC console, I select Create VPC and set a Name tag and IPv4 CIDR block as follows:
In the next step, I create an internet gateway to connect the Internet from the VPC with the following settings. After that, I attach it to the created VPC.
In the next step, I create four subnets(public-spacy-1, public-spacy-2, private-spacy-1, private-spacy-2) with the following settings. Notice that public-spacy-1 and private-spacy-1 have the same availability zone.
Then, I create two NAT gateways and attach it to public subnets to access the Internet from a lambda function in the private subnet. I gave NAT gateways names: NAT spaCy 1 and NAT spaCy 2.
Finally, I create route tables. For public subnets, I add a destination 0.0.0.0/0 and a target Internet Gateway to access the Internet. For private subnets, I add a destination 0.0.0.0/0 and a target NAT spaCy 1 for private-spacy-1 and NAT spaCy 2 for private-spacy-2.
In the EFS console, I select Create file system and make sure that the default VPC and its subnets are selected. For all subnets, I use a security group that allows network access to other resources in the VPC. For simplicity, I set up a security group that allows all traffic.
In the next step, I give the file system a Name tag and select the Next Step.
Then, I select Add access point. I use 1001 for the User ID, Group ID, Owner User ID and Owner Group ID and 750 for the permissions. Also, I limit access to the /models path.
EFS creation is complete. Let’s go to the next step.
In the next step, we publish spaCy as Lambda Layers. We must do the following things:
Install spaCy
Compress it as a Zip file
Publish the file to Lambda Layer
For simplicity, I prepared the following shell script:
You have only to execute it like sh publish_spacy_as_lambda_layer.sh.
Next, we create a Lambda function. In the Lambda console, create a function and select Python 3.7 as the runtime. For permissions, select a role that is attached policiesAWSLambdaVPCAccessExecutionRole and AmazonElasticFileSystemClientReadWriteAccess.
After creating a function, we set up the VPC configuration. Here, we need to specify the same security group and VPC we specified for the EFS mount points and select the private subnets.
Then, we select the Add file system. We select the ESF and access point that has been created before. Here, we set /mnt/models as the local mount point. This is the path where the access point is mounted and corresponds to the /models directory in the EFS.
In the layer section, we select Add a layer to add spaCy.
In the function editor, copy and paste the following code.
Finally, we should increase the memory allocation and timeout values. If the memory allocation is not large enough, the following error will occur.
{  "errorType": "Runtime.ExitError",  "errorMessage": "RequestId: Error: Runtime exited with error: signal: killed"}
As a test, when I test with the data {"text": "He works at Google."}, the following response will be returned.
[  {    "text": "Google",    "label": "ORG",    "start": 12,    "end": 18  }]
New — A Shared File System for Your Lambda Functions
AWS Lambda limits
Natural Language Processing in Cloud
Creating a Serverless NLP API using Spacy and AWS Lambda