Machine Learning Infrastructure with Amazon SageMaker and Terraform — A Case of Fraud Detection | by Quy Tang | Towards Data Science

Think of Machine Learning and its “deep” child, Deep Learning, one would instantly conjure up a web of complexity solvable only by massive computing power. The advent of big data and especially cloud machine learning platforms have vastly simplified access to these computing power for the wider population.
A cloud machine learning platform provides a fully managed infrastructure that automatically scales, integrates with the latest data technology and comes equipped with many built-in algorithms. This enables any developers and data scientists to quickly build and deploy models without worrying about the underlying sophisticated infrastructure.
Amazon SageMaker is the cloud machine learning platform offered by Amazon Web Services (AWS).
The AWS Solutions Builder Team has shared many different solutions built on Amazon SageMaker, covering topics such as predictive analysis in telecommunication or predictive train equipment maintenance.
In this post, I will take the credit card fraud detection solution example and explain how to set up supporting infrastructure with Terraform while breaking down the solution into smaller chunks.
This post targets readers interested in learning about cloud infrastructure to support machine learning and a quick view of how a machine learning workflow looks like.
Disclaimer: I don’t claim any expertise in machine learning. All machine leaning work in this post has been taken from AWS site. My main focus is on Cloud infrastructure and DevOps. I do have data analysis background having used R for financial forecasting and risk management calculations, but that’s not the main purpose here.
The goal of this solution is to predict whether a given credit card transaction is fraudulent or not.
This is a binary (yes/no) classification that typically requires a logistic regression algorithm which, within the context of Amazon SageMaker, is equivalent to the built-in linear learner algorithm with binary classifier predictor type.
The complete project for this article is hosted on Github.
There are 3 main parts to this solution:
Training a model based on a sample credit card transaction dataset.Storing the model in an accessible interface.Using the model to predict whether a generated transaction is fraudulent.
Training a model based on a sample credit card transaction dataset.
Storing the model in an accessible interface.
Using the model to predict whether a generated transaction is fraudulent.
These are typical of a machine learning process, which looks like this:
Let’s follow the implementation based on the above 3 parts.
To train a model, these components are needed:
i. Credit card transaction dataset where fraudulent transactions have already been identified (a.k.a. labeled).
ii. A machine learning algorithm stored as a Docker image (the equivalent of a CD-ROM for the uninitiated).
iii. A server with pre-loaded machine learning libraries for data scientists to prepare, visualize and enhance the training data, and then set the algorithm’s parameters (in layman’s term; hyperparameters in machine learning speak) before starting the training process.
iv. Server(s) to train the model using a particular algorithm, this is essentially the “learning” part of machine learning, which also is the one that requires the most computational power.
Amazon SageMaker provides ii, iii and iv as managed services, while i is taken from a public data source.
This is where it starts to get technical. If you are not interested in infrastructure, you can skip this.
In terms of infrastructure for this part of the solution, we only need to set up iii (called the notebook instance), which contains all necessary libraries and one Jupyter notebook, basically, the data scientists’ tooling code (cum document). The Jupyter notebook, located at source/notebooks/sagemaker_fraud_detection.ipynb in the source code repository, includes code that retrieves ii (algorithm) and delivers iv (training instances) using Amazon SageMaker’s library.
Within AWS world, an instance usually refers to a server, which can be a virtual machine or a container.
As seen above, the notebook instance requires a few supporting resources including:
IAM Role and Policy for the notebook instance:
S3 bucket for storing the Jupyter notebook:
S3 bucket for storing training data as well as generated model data:
SageMaker notebook instance:
In order for the notebook instance to obtain the Jupyter notebook located in our source code, we have to add an init script that downloads the notebook from the aws_s3_bucket.fraud_detection_function_bucket above (we uploaded using the aws_s3_bucket_object). This is done using the notebook instance’s lifecycle configuration:
These are the main steps in the Jupyter notebook:
i. Download sample data and extract features and label (fraud/nonfraud).
ii. Convert the n-dimensional arrays into RecordIO format (a highly efficient data format).
import sagemaker.amazon.common as smacbuf = io.BytesIO()smac.write_numpy_to_dense_tensor(buf, features, labels)
iii. Store the RecordIO data into S3 bucket.
bucket = "fraud-detection-end-to-end-demo"prefix = 'linear-learner'key = 'recordio-pb-data'boto3.resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'train', key)).upload_fileobj(buf)
iv. Retrieve the Docker image for the linear learner algorithm.
container = get_image_uri(boto3.Session().region_name, 'linear-learner')
v. Create a training job with the desired instance type and instance count, change the (hyper)parameters of the algorithm and start training using the training data uploaded to S3 earlier. You can see how simple it is to set up a cluster of servers to train a model and only pay for the time that it takes to train, a major cost saver.
import sagemakers3_train_data = 's3://{}/{}/train/{}'.format(bucket, prefix, key)output_location = 's3://{}/{}/output'.format(bucket, prefix)linear = sagemaker.estimator.Estimator(container,                                                                               get_execution_role(),                                                                        train_instance_count=1,                                                                        train_instance_type='ml.c4.xlarge',                                                                        output_path=output_location,                                                                        sagemaker_session=session)linear.set_hyperparameters(feature_dim=features.shape[1],                                                                         predictor_type='binary_classifier',                                                      mini_batch_size=200)linear.fit({'train': s3_train_data})
Complete code: https://github.com/qtangs/tf-fraud-detection-using-machine-learning/blob/master/source/notebooks/sagemaker_fraud_detection.ipynb
To store a model and present it to the outside world for prediction later on, these components are required:
i. Model data
ii. Other model metadata (container, training job, ...)
iii. An endpoint (interface) configuration
iv. Actual endpoint (a set of servers to run prediction on-demand)
Amazon Sagemaker hosts ii, iii and iv while i (the model data) is stored in S3 as we saw in the previous code (output_location). Both i and ii (the model specification) are created at the end of the call linear.fit(...) .
The endpoint configuration (iii) and the actual endpoint (iv) are created by this part of the Jupyter notebook:
linear_predictor = linear.deploy(initial_instance_count=1,                                                            endpoint_name="fraud-detection-endpoint",                                                            instance_type='ml.m4.xlarge')
Complete code: https://github.com/qtangs/tf-fraud-detection-using-machine-learning/blob/master/source/notebooks/sagemaker_fraud_detection.ipynb
With the created endpoint, we can now use it on any generated transaction.
To simulate transactions coming in for prediction, we need:
a CloudWatch Event is created to trigger Lambda function every 1 minute.
an AWS Lambda function is created using code located at source/fraud_detection/index.py in the code repository. It follows these steps: randomly selects a predefined fraud or nonfraud transaction, sends the transaction to the endpoint to obtain a fraud prediction, sends the result to a Kinesis Data Firehose after some minor processing.
a Kinesis Data Firehose that stores streamed results into S3.
and finally, the S3 bucket.
As seen above, the resources with the blue Terraform logo are required:
CloudWatch event:
IAM Role and Policy for Lambda prediction function:
S3 bucket for storing the function code, taken from source/fraud_detection/index.py in our code repository
The Lambda function which processes new transactions:
IAM Role and Policy for the Kinesis Data Firehose:
S3 bucket to store processed transactions from the Kinesis Data Firehose:
And finally, the Kinesis Data Firehose:
Terraform code is located in the folder terraform (original CloudFormation can be found in cloudformation).
Before running terraform apply, there are a few set-ups to perform:
i. Copy terraform_backend.tf.template to terraform_backend.tf and modify values accordingly. You need to manually create an S3 bucket or use an existing one to store the Terraform state file.
Using remote backend for Terraform state is a good practice. It should be practiced even for a simple scenario like this.
ii. Copy terraform.tfvars.template to terraform.tfvars and modify values accordingly. You don’t need to create any buckets specified in here; they’re to be created by terraform apply.
iii. Once the above files are created, simply run through the following terraform commands. Remember to ensure all commands return ok and to review the terraform plan before applying.
# Set default AWS profile,# use 'set' instead of 'export' for Windows.export AWS_PROFILE=<your desired profile>terraform initterraform validateterraform plan -out=tfplanterraform apply --auto-approve tfplan
The Terraform plan output should look like this:
And the result of terraform apply:
Once all Terraform resources are set up, you need to follow these manual steps as documented by the AWS site:
Run the Notebook
Navigate to the Amazon SageMaker console.In the navigation pane, select Notebook instances.Select FraudDetectionNotebookInstance.The notebook instance should already be running.Select Open Jupyter.In the Jupyter notebook interface, open the sagemaker_fraud_detection.ipynb file.In the Cell dropdown menu, select Run All to run the file.
Navigate to the Amazon SageMaker console.
In the navigation pane, select Notebook instances.
Select FraudDetectionNotebookInstance.
The notebook instance should already be running.
Select Open Jupyter.
In the Jupyter notebook interface, open the sagemaker_fraud_detection.ipynb file.
In the Cell dropdown menu, select Run All to run the file.
Enable the CloudWatch Events Rule
Navigate to the AWS Lambda console.In the navigation pane, select Functions.Select the fraud_detection_event_processor Lambda function.In the diagram in the Designer tab, select CloudWatch Events.In the CloudWatch Events tab, select <stackname>-ScheduleRule-<id>.Select Actions > Enable.Select Enable.
Navigate to the AWS Lambda console.
In the navigation pane, select Functions.
Select the fraud_detection_event_processor Lambda function.
In the diagram in the Designer tab, select CloudWatch Events.
In the CloudWatch Events tab, select <stackname>-ScheduleRule-<id>.
Select Actions > Enable.
Select Enable.
Verify the Lambda Function Is Processing Transactions
Navigate to the AWS Lambda console.In the navigation pane, select Functions.Select the fraud_detection_event_processor Lambda function.Select Monitoring and verify that the Invocations graph shows activity.After a few minutes, check the results Amazon S3 bucket for processed transactions.
Navigate to the AWS Lambda console.
In the navigation pane, select Functions.
Select the fraud_detection_event_processor Lambda function.
Select Monitoring and verify that the Invocations graph shows activity.
After a few minutes, check the results Amazon S3 bucket for processed transactions.
Once you are done with the experiment, simply perform the followings to delete all resources and save costs. Again, remember to review the terraform plan before applying.
terraform plan -destroy -out=tfplanterraform apply tfplan
You should be able to see output like this:
Amazon SageMaker is a powerful platform, we have just barely touched its surface.
Thanks to the AWS Solution Builders Team, we have many fantastic examples of how Amazon SageMaker can be utilized.
At the same time, Terraform simplifies the infrastructure setup for an enterprise setup where DevOps and Continuous Delivery become more and more important.
In future posts, I will look into more complicated examples provided by the AWS Team and will also explore how Terraform infrastructure can be designed to fit into a CI/CD pipeline within a machine learning setup.
Thank you for reading. Do comment below to share your thoughts.
The complete project for this article is hosted on Github.
Some rights reserved