Using AWS Sagemaker and Lambda to Build a Serverless ML Platform | by Andrewngai | Towards Data Science

Most data enthusiasts know how to build and train a model, but how to deploy your model and make it useful in real-life sometimes can be a challenging issue for beginner data scientists. Luckily, there are many different platforms and tools available to help with model deployment. Amazon Sagemaker is one of my favorites, as it largely reduces the effort and hesitation of building, training, and deployment of your models. With the help of numerous AWS functionalities and tools such as Lambda function, S3, Dynamo DB, the entire process of building a working ML application can be at the click of a mouse.
In this article, I would like to demo how we can leverage the power of AWS to build a serverless ML application that predicts air flight delay.
Dataset
Demo Architecture
Create a Notebook Instance
Create a Sagemaker Endpoint
Create a Lambda Function
Create API Endpoints
End to End Testing
Conclusion
The dataset is from the U.S. Department of Transportation contains 7.21 million flight records in 2018 with 28 columns.
https://www.kaggle.com/yuanyuwendymu/airline-delay-and-cancellation-data-2009-2018
Due to a large amount of data(7.21million), we used Google Bigquerry for data cleaning, preprocessing and simple feature engineering. As the purpose of this project is to demonstrate how AWS helps with model training and deployment, We will not spend to much time on how we pre-process the data with Bigquerry(Maybe a future topic). The processed data ready for training is then stored into an S3 bucket in CSV format. Now we can start building our demo on AWS
To make this application useful, we will take advantage of AWS API Gateway and Lambda function to build an API taking HTTP POST request that contains airline information(a real-life example could be an APP or website that takes Airline number as input), the POST request will trigger the Lambda function to parse the value and send test data to a Sagemaker endpoint that have the model deployed. The return value will be parsed by Lambda function again and send the prediction result back to the user. The architecture diagram demonstrates the end to end pipeline from user input to prediction output.
As we mentioned earlier, the pre-processed result is stored in an S3 bucket for easy usage. However, Sagemaker can consume data from different channels. For example, Dynamo DB, Aurora or even IoT device data from AWS IoT. Sagemaker also provides various tools for automatic model tuning and selection. Let’s get started by first create a Jupyter notebook instance in Sagemaker. From the AWS main page, choose service and go into Sagemaker. And you will see the following options, choose Notebook instances.
If you have never used Amazon SageMaker before, for the first two months, you are offered a monthly free tier of 250 hours of t2. medium or t3. medium notebook usage for building your models, plus 50 hours of m4. Enter the name and instance type for your notebook. If you have never created an IAM role before create a new role or use the default Sagemaker IAM role. IAM role controls the resource permission for your instances.
Amazon SageMaker has built-in Jupyter Notebooks that allow you to write code in Python, Julia, R, or Scala. We will be using Python 3 for this project. It also provides a list of sample notebooks that loaded, when the notebook instance spins up. Each sample notebook has different use cases and contains detail comments on each step. I strongly recommend beginners to go through those notebooks and you will have a much better idea of how to utilize Sagemakers.
Now let’s create our notebook, the notebook is modified based on the Breast Cancer Prediction.ipynb. You can read through the original sample. Remember to change the S3 bucket name and CSV file to match yours.
The last line of code will delete the endpoint that is created from the notebook, let’s first comment out the last line. Then you can execute the notebook by hit Run All or use the Shift and Enter keys to run the cell step by step. By running the notebook, the model is trained and deployed with a Sagemaker endpoint created. You can view this endpoint from the Amazon SageMaker console. you can change the default endpoint name to be something more meaningful.
Now we have a SageMaker endpoint. Let’s create a Lambda function to call the endpoint. It will parse the HTTP POST request, revoke the Sagemaker endpoint, return prediction, parse the result and send it back to users. The Lambda can use boto3 sagemaker-runtime.invoke_endpoint() to call the endpoint
AWS Lambda is a useful tool, allowing the developer to build serverless function on a cost per usage-based. You also benefit from the faster development, easier operational management, and scalability of FaaS. From the Lambda function, select Author from scratch. Enter your function name, choose Python 3.6 for this project. Remember to choose an IAM execution role with a policy that gives your function permission to invoke sagemaker endpoint.
Here is the code for our Lambda function, it uses the ENDPOINT_NAME as an environment variable that holds the name of the SageMaker endpoint. Remember to assign the variable to your endpoint name under the environment variables section.
import osimport ioimport boto3import jsonimport csv# grab environment variablesENDPOINT_NAME = os.environ['ENDPOINT_NAME']runtime= boto3.client('runtime.sagemaker')def lambda_handler(event, context):    print("Received event: " + json.dumps(event, indent=2))        data = json.loads(json.dumps(event))    payload = data['data']    print(payload)        response = runtime.invoke_endpoint(EndpointName=ENDPOINT_NAME,                                       ContentType='text/csv',                                       Body=payload)    print('res is !!!!')    print(response)    result = json.loads(response['Body'].read().decode())    print('result is!!!!')    print(result)    pred = int(result['predictions'][0]['score'])    #pred = int(result['predictions'][0])    predicted_label = 'delay' if pred == 1 else 'no delay'        return predicted_label
Now we have the Lambda function ready, let’s create the API to receive HTTP requests and integrate everything. You can follow the AWS Documentation to create the API from API Gateway. Or just follow these simple steps.
Open the AWS API Gateway console. Choose Create API Choose an API name, you can leave the Endpoint Type as Regional. Choose Create API.
Open the AWS API Gateway console. Choose Create API Choose an API name, you can leave the Endpoint Type as Regional. Choose Create API.
2. Next, create a Resource choosing from the Actions drop-down list, When the resource is created, from the same list, choose to Create Method to create a POST method.
3. On the screen that appears, enter the following: For Integration type, choose Lambda Function. For Lambda Function, enter the function you created
4. After the setup is complete, deploy the API to a stage. From the Actions drop-down list, choose Deploy API.
5. On the page that appears, create a new stage. and choose the Deploy This step will give you the Invoke URL.
Now we have built a serverless ML pipeline that can take user’s data from HTTP POST and return ML prediction results from the model we deployed. We can test it on Postman. Postman is an interface testing tool. When doing interface testing, Postman is like a client. It can simulate all kinds of HTTP requests initiated by users, send the request data to the server, obtain the corresponding response result, and verify whether the result data in the response matches the expected value. Now we send a POST request to the Invoke Url we just created in the last step, it triggers the Lambda function and invokes our Sagemaker endpoint to make the prediction. We can see we get a result back as ‘no dealy’
Concluding thoughts
AWS Sagemaker has been a great tool for all-level data scientists and ML application developers to achieve robust and complete end-to-end ML solutions. It can automate many tedious efforts like hyperparameter tuning, model selection, even data labeling. With the help of the Lambda function, ML application can be highly flexible and cost-effective.
Thanks for reading and I am looking forward to hearing your questions and thoughts. If you want to learn more about Data Science and Cloud Computing, you can find me on Linkedin.