Deployment of Machine learning Models Demystified (Part 1) | by Bamigbade Opeyemi | Towards Data Science

This is the first part of a two-part series. you can read the second part after this
Imagine building a supervised machine learning(ML) model to decide whether a loan application should be approved. With the model confidence level (probability) in successful applications, we can calculate the risk-free loanable amount. The deployment of such ML-model is the goal of this project.
Join me as I walk you through my internship project at Data Science Nigeria where I have been opportune to work alongside crème de la crème of Data Scientists, Software Engineers and Researchers in Artificial intelligence. The goal is to raise 1 million Ai talent in 10 years which am proud to be part of.
Whereas data scientists build Machine learning models in jupyter lab, google colab and the likes, Machine learning engineers take the built model into production.
Deployment of an ML-model simply means the integration of the model into an existing production environment which can take in an input and return an output that can be used in making practical business decisions.
To demystify the deployment process, I created this post which is split into four chapters. In chapter 1, we cover the process of building a baseline model and in chapter 2, we adopt this model intro production-ready code. Chapters 3 and 4 are covered in Part 2 of this series.
Chapter 1: Building a base-line model
A base-line model in this context refers to an ML-model having the minimum possible number of features but with good evaluation measures. When building an ML-model for deployment purposes, you must always have your end-users in mind. A model with thousands of features to attain an accuracy 90+% on evaluation might not be good enough for deployment for several reasons:
Portability: Is the ability of software to be transferred from one machine or system to another. A portable model decreases the response time of your code and the amount of code you have to rewrite when your goals change will be minimal.
Scalability: Is the ability of a program to scale. A model is considered scalable when it doesn’t need to be redesigned to maintain effective performance.
Operationalization: Refers to the deployment of models to be consumed by business applications to predict the target value of a classification/regression problem.
Test: Refers to the validation of output to processes and input. A robust model will be unnecessarily difficult to test.
Click to download the dataset that will be used for this project, the notebook and the exclusive summary that contain Exploratory Data Analysis, Data Preprocessing, Feature Engineering, Feature Selection, Training the ML model and Model Evaluation.
As a first step, we use pandas to import our test and training data.
train = pd.read_csv("train.csv")test = pd.read_csv("test.csv")
Data Preprocessing: This involves the transformation of raw data into an understandable format suitable to train a model. It entails taking care of missing values, outliers, encoding categorical variables.
For the purpose of deployment, you want to always keep a record of the values you are using to fill the missing values and the approach has to be replicable. Features having missing values can be seen at a glance with the code snippets below
There are many ways you can take care of the missing values in numerical features but to keep things simple, I will experiment with the mean and the median.
For each of the numerical features with missing values, you compute the mean and the median, impute this value and fit a model on the data to see the effect for each case on the validation sets. For instance, experimenting with the applicant_income feature, we can define a simple python function that takes in the dataframe, variable of interest, mean and median and fill the missing parts of the features with the mean and median.
A simple feature engineering that can be done when filling missing values is to create a binary feature that indicates missingness (0 or 1) and you can tell if the missingness has a positive predict effect or not.
#addition missingness features# create variable indicating missingness in applicant_incomeX_train['applicant_income_NA'] = np.where(X_train.applicant_income.isnull(),1,0)X_test['applicant_income_NA'] = np.where(X_test.applicant_income.isnull(),1,0)
Now, we can use a simple logistic regression model to determine the best approach to filling missing values.
This approach is repeated for all the features with missing values and the best decision is made based on the model outcome.
Feature Engineering: this is the process of using domain knowledge of the data to create features that make machine algorithms work. The only feature I will create is income_loan_ratio to avoid complexity during deployment.
Feature Selection: Is the process where you automatically or manually select those features which contribute most to your prediction variable. Having irrelevant features in your data can decrease the accuracy of the models.
There are several methods of selecting features for the final model which include
Filter methods such as constant features elimination, quasi-constant features elimination, duplicate feature elimination, fisher score, univariate method, mutual information, correlation, etc.
Wrapper methods such as step forward selection, step backward selection and exhaustive search etc.
Embedded methods such as LASSO, Decision tree derived importance, Regression coefficients etc.
Training the ML model: the type of machine learning algorithm for this project is supervised and it is a classification model. Before training the model, we need to separate the features from the target variable.
I have chosen to use GradientBoostingClassifier from sklearn for deployment but other algorithms such as xgboost, Microsoft LGBM were also explored during the experiment.
Model Evaluation
From the output of the trained model above, the accuracy on test data is 70.3% with ROC-AUC of 77.2%. Since the goal of this project is to predict the acceptance status of mortgages applications and also to predict the safe amount on accepted loan applications, 70% accuracy is good enough for the problem case Scenario. The model can be improved later but we will not be focusing on that because we want to make deployment as simple as possible for now.
Chapter 2: Adopting an ML System Architecture and Writing Production-Ready Codes
According to ISO/IEC 42010 and I paraphrase, Architecture can be defined as the way software components are arranged and the interactions between them.
ML system architecture refers to the way in which the software components that makeup system are arranged and the interaction between them in achieving a predefined goal. The development and deployment of ML-models are relatively fast and cheap but maintaining them effectively over time is difficult and expensive. Depending on your company’s capacity and goal, choosing the best architecture might be a bit challenging as it requires the synchronisation of the business, data science, engineering and DevOps.
This is the first part of a two-part series. you can read the second part after this
When designing an ML system architecture, the following should be borne in mind.
Modularity: the code used in preprocessing/feature engineering should be arranged in comprehensive pipelines.
Reproducibility: the output of each component must be replicable for any version in time.
Scalability: the model must be serveable to a large number of customers with minimal response time.
Extensibility: it should be easy to modify for future tasks.
Testing: the ability to test variation between model versions.
Automation: eliminating manual steps wherever possible to reduce error chances.
Overview of ML architectures
Train by batch, predict on the fly, serve via REST API: training and persisting are done offline while prediction is done in real-time.
Train by batch, predict by batch, serve through a shared database: training and persisting are done offline while predictions are done in a distributed queue which is almost similar to a real-time prediction
Train, predict by streaming: both training and predicting are done on different but connected streams.
Train by batch, predict on mobile(or other clients): similar to type 1 but the prediction is made on customer gadget.
Architecture Comparison
For this project, pattern 1 will be used and the predictions can be single or in batches, i.e we will be training the model locally and expose the predict function via flask where we can upload our test batch for prediction.
In the figure above, it can be seen that the whole system is divided into development and production. in the development, the Training Data, Feature Extractor and Model Builder section are done offline while production is online.
Training Data: this unit deals with the extraction of the dataset to be used in training the model into the training environment and formatting for training. this unit can be complex or simple depending on the ML-architecture you are working on.This can include pulling of data from multiple Sequels, Hadoop distributed file system or make API calls, for the purpose of this project, a simple python script will be written to fetch the dataset already downloaded into a folder/directory to the training environment and then passed through a data pipeline where several preprocessing will be done before training.
Feature Extractor: this is the unit that selects/generates essential features needed in training the model. note that this is part of the pipeline uses sklearn. apart from the sklearn library, another popular library that can be used to achieve the same task is TensorFlow Extended(TFX)which is out of the scope of this article.
Model Builder: this is the unit where serializing and persisting of the trained models, versioning and formatting for deployment. In python context, this can be the unit that involves in package building using setuptools or simply saving the model in a pickle format depending on the architecture you adopt. there is a huge difference between these approaches but will not be discussing it in this article. though I experimented with both approaches but will be sharing with you the latter approach so as to keep things simple.
Trained model: this is the output of the three sub-topics discussed above which can be easily deployed via REST API.
This is the first part of a two-part series. you can read the second part after this
In production, a request is sent via REST API in batches, cleaned and preprocessed by the Feature Extractor, then the loaded trained model is used in predicting if the loan application is accepted or not. the accepted loans are then further analysed for risk-free loanable amount before returning the prediction through our end-point.
In Part 2(chapters 3 & 4), I will be discussing how to convert the notebook into production ready-code, and how to create a reproducible pipeline, API endpoints, testing the end-points through Postman, meaning and use of CI/CD in ML-System Architectures, Deploy to a Pass, Docker Container, Deploying to Lass, Serverless Deployment etc.
This is the first part of a two-part series. you can read the second part after this
You can clone the Github repository via the link below:
github.com
There are lots of ways to deploy ML-models into production, numerous ways to store them and different ways to manage predictive models after deployment. choosing the best approach to a use case can be challenging but with the team’s technical and analytics maturity, the overall organization structure and its interactions can help in selecting the right approach for deploying predictive models to production.
Connect with me on Twitter and LinkedIn
Do find time check out my other articles and further readings in the reference section. Kindly remember to follow me so as to get notified of my publications.
medium.com
heartbeat.fritz.ai
heartbeat.fritz.ai
heartbeat.fritz.ai
heartbeat.fritz.ai
heartbeat.fritz.ai
medium.com
References
[1] Julien Kervizic, Overview of the different approaches to putting Machine Learning (ML) models in production.
[2] Mark Hornick, Data Science Maturity Model — Deployment (Part 11)
[3] Soledad Galli & Christopher Samiullah, Deployment of Machine Learning Models
Special thanks to my mentors (Dr.Olubayo Adekanmbi, James Allingham ), wuraola oyewusi and the entire staff of Data Science Nigeria.