Machine Learning Model Development and Deployment with MLflow and Scikit-learn Pipelines
By: Elizabeth Obee
Tags: Machine Learning Python, Mlflow, Deployment, Scikit Learn, Pipeline

Pipelines

Pipelines have a number of advantages.

Ensuring no leakage of information between test and train data

Bundling of models with preprocessing steps into a single serialized object instead of multiple steps and outputs

Allowing for the same data preparation steps to run during development and deployment

The Scikit-learn pipeline is also compatible with other modeling packages such as Keras and XGBoost.

For this pipeline we will import the Cleveland Clinic heart disease dataset from the UC Irvine Machine Learning Repository. A very similar dataset is also available on Kaggle. Features related to laboratory and physiological testing include:

Age of patient

Sex of patient

CP — chest pain type

Trestbps — resting blood pressure

Chol — serum cholesterol

FBS — fasting blood sugar

Restecg — resting electrocardiogram results

Thalach — maximum heart rate

Exang — exercise induced angina

Oldpeak — ST depression induced by exercise

Slope — slope of the peak exercise ST segment

CA — number of major vessels colored with flouroscopy

Thal — type of defect

Num — diagnosis of heart disease

Let’s read in the data, set up our target variable, and take a look at the data.

import numpy as np

import pandas as pd

from sklearn.compose import ColumnTransformer

from sklearn.pipeline import Pipeline

from sklearn.impute import SimpleImputer

from sklearn.preprocessing import StandardScaler, OneHotEncoder

from sklearn.linear_model import LogisticRegression

from sklearn.model_selection import train_test_split

from sklearn.metrics import accuracy_score, precision_score, recall_score import mlflow

import mlflow.sklearn

from mlflow.models.signature import infer_signature URL = 'https://archive.ics.uci.edu/ml/machine-learning-databases/heart-disease/processed.cleveland.data' df = pd.read_csv(URL, header=None, names=['age','sex','cp','trestbps','chol','fbs','restecg','thalach'

,'exang','oldpeak','slope','ca','thal','num']) df['target']=np.where(df['num'] > 0,1,0) df.head()

In order to demonstrate the use of a custom transformer we will create a new feature based upon the ratio of the resting blood pressure to the maximum blood pressure. This feature will be created as a new class and saved into a separate file so it can be output to the MLflow tracking server to be used during deployment in combination with the saved model.

import pandas as pd

from sklearn.base import BaseEstimator, TransformerMixin #Custom Transformer Class

class NewFeatureTransformer(BaseEstimator, TransformerMixin):

def fit(self, x, y=None):

return self def transform(self, x):

x['ratio'] = x['thalach']/x['trestbps']

x=pd.DataFrame(x.loc[:, 'ratio'])

return x.values

Now, let’s split our data up into train, validation, and test datasets.

train, test = train_test_split(df, test_size=0.2)

train, val = train_test_split(train, test_size=0.2) print(len(train), 'Train Examples')

print(len(val), 'Validation Examples')

print(len(test), 'Test Examples')

We are now ready to import our custom class and define the pipeline.

from custom_class import NewFeatureTransformer #Define the Pipeline numeric_features = ['age','trestbps','chol','thalach','oldpeak']

numeric_transformer = Pipeline(steps=[

('imputer', SimpleImputer(strategy='median')),

('scaler', StandardScaler())]) categorical_features = [ 'cp','restecg','ca','thal','slope']

categorical_transformer = Pipeline(steps=[

('imputer', SimpleImputer(strategy='constant',fill_value=0)),

('onehot', OneHotEncoder(handle_unknown='ignore'))]) binary_features = [ 'sex','fbs','exang']

binary_transformer = Pipeline(steps=[

('imputer',SimpleImputer(strategy='constant',fill_value=0))]) new_features_input = [ 'thalach','trestbps']

new_transformer = Pipeline(steps=[

('new', NewFeatureTransformer())]) preprocessor = ColumnTransformer(

transformers=[

('num', numeric_transformer, numeric_features),

('cat', categorical_transformer, categorical_features),

('binary', binary_transformer, binary_features),

('new', new_transformer, new_features_input)]) # Now join together the preprocessing with the classifier. clf = Pipeline(steps=[('preprocessor', preprocessor),

('classifier', LogisticRegression())], verbose=True) #fit the pipeline

clf.fit(train, train['target'].values) #create predictions for validation data

y_pred = clf.predict(val)

Now that we have our pipeline, the next step is to save the resulting model to the MLflow tracking server.

MLflow

The MLflow Project is a framework-agnostic approach to model tracking and deployment, originally released as open source in July 2018 by Databricks. MLflow is now a member of the Linux Foundation as of July 2020. It is also possible to deploy models saved on a MLflow tracking server via Seldon into Kubernetes. This allows the usage of the model tracking functionality of MLflow with the expanded deployment capabilities of Seldon.

Although MLflow has a scikit-learn “flavor” for models, due to the usage of a custom transformer we will need to instead use the generic “python function flavor”. Usage of MLflow with scikit-learn pipelines allows us to save the model definition and all preprocessing steps and transformations into a single object, plus our custom transformer code. First set up the model output class and the conda environment that is needed to execute the model in the future.

class ModelOut (mlflow.pyfunc.PythonModel):

def __init__(self, model):

self.model = model def predict (self, context, model_input):

model_input.columns= map(str.lower,model_input.columns)

return self.model.predict_proba(model_input)[:,1] mlflow_conda={'channels': ['defaults'],

'name':'conda',

'dependencies': [ 'python=3.6', 'pip',

{'pip':['mlflow','scikit-learn','cloudpickle','pandas','numpy']}]}

Next we will output the model and some metrics to the tracking server.

with mlflow.start_run():

#log metrics

mlflow.log_metric("accuracy", accuracy_score( val['target'].values, y_pred))

mlflow.log_metric("precison", precision_score( val['target'].values, y_pred))

mlflow.log_metric("recall", recall_score( val['target'].values, y_pred)) # log model

mlflow.pyfunc.log_model( artifact_path="model",

python_model=ModelOut(model=clf,),

code_path=['custom_class.py'],

conda_env=mlflow_conda)

signature = infer_signature(val, y_pred) #print out the active run ID

run = mlflow.active_run()

print("Active run_id: {}".format(run.info.run_id))

Execution Using MLflow Project

Using MLflow Project and Github, the model saved to the tracking server can be linked to the Github code. You can also use the MLproject file to execute locally. Here is an example of what our very simple MLproject file looks like.

name: heartdisease_pipeline_with_mlflow conda_env: disease_conda.yaml entry_points:

main:

command: 'python pipeline_with_mlflow.py'

We can either execute this based upon the conda.yaml file to set up a new conda environment, or execute with the — no-conda option.

Now we can review the details of our run_id in the MLflow tracking server, by executing mlflow ui at the anaconda prompt, of our activated environment that contains MLflow. We find the tracking server at http:\\localhost:5000

Model Serving

The model can now be served from the Anaconda prompt.

mlflow models serve -m file:C:\Users\Elizabeth\Documents\GitHub\EO\mlruns\0\4c5e296e18174521a470891251eed5ea\artifacts\model\

You will get a message, after building a conda environment and activating it, that the model is serving on http://localhost:5000. We can now provide data to the model and receive a prediction. We will use python here to demonstrate but this API could also be invoked to score data from a stream.

import requests

host = 'localhost'

port = '5000' url = f'http://{host}:{port}/invocations' headers = {'Content-Type': 'application/json',}

# test contains our data from the original train/valid/test split http_data = test.to_json(orient='split')

r = requests.post(url=url, headers=headers, data=http_data) print(f'Predictions: {r.text}')

Summary

Writing python based production ready deployment code for data science models is facilitated by the usage of Scikit-learn pipelines and MLflow model tracking servers. With this method the data scientist can produce an API from a model without additional ETL of an input data stream. This can facilitate deployment and allow software engineers to focus upon infrastructure and systems integration, including continuous integration and delivery instead of maintaining a modeling related code base. It also makes redeployments as easy as referencing a new run_id on the MLFlow tracking server.